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1. INTRODUCTION 



This report is the fourth in a series describing the implementation of 
MDBS, a multi-backend database system [Kerr82, He82, Boyn83] . The original 
design was given in [Hsia81a, Hsia81b] . It is assumed that the reader is 
already familiar with these earlier reports. We will, however, give a very 
brief review of the MDBS design. 

An overview of MDBS hardware organization is shown in Figure 1. MDBS is 
connected to a host computer through the controller. The controller and back- 
ends are, in turn, connected by a broadcast bus. The controller receives 
requests from a host computer. It then broadcasts each request to all back- 
ends at the same time over the bus. Since the database is distributed across 
the backends, a request can be executed by all backends in parallel. 

To manage the database (often referred to as user data) , MDBS uses direc- 
tory data. Directory data in MDBS corresponds to attributes, descriptors, and 
clusters. An attribute is used to represent a category of the user data; 
e.g., SALARY is an attribute that corresponds to actual salaries stored in the 
database. A descriptor is used to describe a range of values that an attri- 
bute can have; e.g., (10001 <= SALARY <= 15000) is a possible descriptor for 
the attribute SALARY. The descriptors that are defined for an attribute, 
e.g., salary ranges, are mutually exclusive. Now the notion of a cluster can 
be defined. A cluster is a group of records such that every record in the 
cluster satisfies the same set of descriptors. For example, all records with 
SALARY between $10,001 and $15,000 may form one cluster whose descriptor is 
the one given above. In this case, the cluster satisfies the set of a single 
descriptor. In reality, a cluster tends to satisfy a set of multiple descrip- 
tors. 



The process structure of MDBS is shown in Figure 2. A major design goal 
for MDBS was to minimize the work done by the controller and to maximize the 
work done by the backends. The controller must, however, perform some func- 
tions. It must first prepare a request for execution by the backends. This 
function is performed by request preparation. The controller must also coordi- 
nate responses from the backends. This function is performed by post process- 
ing. In addition, for consistency reasons, certain functions required for 
record insertion must also be performed in the controller. These functions are 
performed by insert information generation. 
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Figure 1. The MDBS Hardware Organization 
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Figure 2. The MDBS Process Structure 



As much work as possible has been given to the backends, this work con- 
sists of three categories of functions: directory management, concurrency con- 
trol and record processing. The directory management functions are used to 
determine the addresses of the records required to process a particular 
request. The concurrency control function allows concurrent access to the 
database by different requests. The record processing functions perform the 
actual data retrieval and storage as well as the processing required on any 
particular record (e.g., the computation of a maximum value) . 

1^._1. The Implementation Strategy 

In this section we provide the reader with a brief review of the MDBS 
implementation strategy. Recall that the implementation strategy involved the 
development of MDBS in seven versions, labeled version A to version G. Ver- 
sion A was the initial version where the controller and backend functions were 
implemented on a single minicomputer. Version A was implemented on a VAX- 
11/780 running the UNIX operating system. It included the request preparation 
and insert information generation functions of the controller and the direc- 
tory management and record processing functions of the backends. The post- 
processing function was not implemented. Instead, the output was displayed 
directly from record processing. The disk input/output routines were omitted 
since they were operating-system-dependent, and subsequent versions of MDBS 
would have the PDP-ll/44s (running RSX-11M) as the actual backends. Since the 
database was not to be stored on disks in this version, we implemented a 
pseudo-disk using the main memory. In addition, an interactive test interface 
was implemented. 

Version B involved the development of a multi -process, multi-computer 
system with the same functionality as version A. The controller had three 
processes, request preparation, insert information generation, and post- 
processing. The backend had two processes, directory management and record 
processing. Concurrency control was added as a third process in a later ver- 
sion. Two computers were used, a VAX-11/780 (running VMS) for the controller, 
and a PDP-11/44 (running RSX-11M) for the backend. 

Because of the multi-process, multi-computer structure, a message-passing 
facility was designed. Three types of message-passing facilities were 
defined: message passing within the controller, message passing within the 
backend, and message passing between the computers. The first two types are 
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categorized as intra-computer message passing, the last type as inter-computer 
message passing. The original explanation of the MDBS message passing facil- 
ity can be found in [Boyn83]. The revised definitions of the MDBS messages is 
contained in Chapter 5 of this report. 

As just described, version B used only a single backend. Thus we con- 
verted to two backends for version C. This version ran on three computers, a 
VAX-11/780 and two PDP-ll/44s. However, it still lacked several required func- 
tions. There was no concurrency control. In addition, all the data including 
the database and its directory, were still stored in primary memories. Thus no 
disk input and disk output was required. 

By changing from using a simulated disk in version C to an actual disk 
system, we obtained version D. This change, though logically simple, was dif- 
ficult to implement, since it required the development of a low-level inter- 
face with the operating system of the PDP-ll/44s. This interface was dis- 
cussed in [Boyn83]. Version D included all the functions we had intended for 
our first real system, except concurrency control. Thus we next added a con- 
currency control process to give us version E. This process was also described 
in [Boyn83] . 

The next step in our implementation, version F, was to change directory 
management so that directory information is stored on the secondary memory 
rather than in the primary memory. This change is complex, since restructuring 
of the directory data is also required. The secondary-memory-based directory 
management of version F is described in [Boyn83] . 

The final version, version G, will incorporate access control in the 
backends and a friendly user-interface in the controller or host computer. 

_1 .2. Concurrency Control Revisited 

The main focus of this report is on the modification of the concurrency 
control process. Recall that directory data is used for the fast, efficient 
access of user data. In order to maintain both the consistency and integrity 
of the user data, we must also control access to directory data. Conse- 
quently, the concurrency control process developed in version E for control- 
ling access to user data, must be expanded to include controlled access to 
directory data. In the rest of this report we describe in detail the imple- 
mentation of version F, the multi-computer MDBS with concurrency control for 
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directory data. Chapter 2 contains an analysis of the concurrency control 
process for directory data. Chapter 3 describes the improvements made in the 
request execution of an update request. Chapter 4 describes the changes in the 
structure of directory management caused by the use of the secondary storage 
for directory data. Chapter 5 presents the revised definitions of the MDBS 
messages. Finally, Chapter 6 concludes this series of reports [Kerr82, He82, 
Boyn83] on the implementation of MDBS and presents a brief discussion of the 
next phase in the development of MDBS. 
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2. CONCURRENCY CONTROL IN DIRECTORY MANAGEMENT 



In this chapter we discuss the concurrency control process for directory 
data. That is, we consider just how the access to attributes, descriptors, 
and cluster definitions must be controlled to preserve the consistency and 
integrity of the database. To motivate this discussion, some background infor- 
mation is presented. 

MDBS is designed to perform the primary database operations, INSERT, 
DELETE, UPDATE, and RETRIEVE. Users access MDBS through the host by issuing 
either a request or a transaction. A request is a primary operation along 
with a qualification. A qualif ication is used to specify the information of 
the database that is to be accessed by the request. There are four types of 
requests, corresponding to the four primary database operations. An example of 
an update request would be: 

UPDATE (FILE=Census and CITY=Cumberland) <POPULATION=40000> 

which sets the population of Cumberland to 40,000. Notice that the qualifica- 
tion component of an update request consists of two parts, the query 
( (FILE=Census and CITY=Cumberland) ) and the modifier (CITY=Cumberland) . The 
query specifies which records of the database are to be updated. The modifier 
specifies how the records satisfying the query are to be updated [HsiaSla] . A 
user may wish to treat two or more requests as a transaction . In this situa- 
tion, MDBS executes the requests of a transaction without permuting them, 
i.e., if T is a transaction containing the requests <R1XR2>, then MDBS exe- 
cutes the request R1 before request R2. Finally, we define the term traffic- 
unit to represent either a single request or a transaction in execution. 

We recall that the directory information is stored in three tables: the 
attribute table (AT) , the descriptor-to-descriptor-id table (DDIT) and the 
cluster-definition table (CDT) . The attribute table maps directory attributes 
to the descriptors defined on them. A sample AT is depicted in Figure 3a. The 
descriptor-to-descriptor-id table maps each descriptor to a unique descriptor 
id. A sample DDIT is given in Figure 3b. The cluster-definition table maps 
descriptor-id sets to cluster ids. Each entry consists of the unique cluster 
id, the set of descriptor ids whose descriptors define the cluster, and the 
addresses of the records in the clusters. A sample CDT is shown in Figure 3c. 
Thus, to control access to directory data, we must control access to the AT, 
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Figure 3c. A Sample Cluster-Definition Table (CDT) 
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DDIT, and CDT. 



Lastly, we identify three classifications of descriptors. A type- A 
descriptor is a conjunction of a less-than-or-equal-to predicate and a 
greater-than-or-equal-to predicate, such that the same attribute appears in 
both predicates. An example of a type-A descriptor is as follows: 

((POPULATION >= 10000) and (POPULATION <= 15000)). 

A type- B descriptor consists of only an equality predicate. An example of a 
type-B descriptor is: 



(POPULATION = 17000) . 

Finally, a type- C descriptor consists of the name of a type- C attribute . The 
type-C attribute defines a set of type-C sub-descriptors. Type-C sub- 
descriptors are equality predicates defined over all unique attribute values 
which exist in the database. For example, the type-C attribute CITY forms the 
type-C sub-descriptors 



(CITY=Cumberland) , (CITY=Columbus) 

where "Cumberland" and "Columbus" are the only unique database values for 
CITY. 



In the remainder of this chapter we first consider just why concurrency 
control for directory data is needed in MDBS. Then we examine the concurrency 
control process in the descriptor search phase. Lastly, we describe the con- 
currency control process in the cluster search phase. 

2^_1 . The Need for Concurrency Control in Directory Management 

To understand the need for controlling access to directory data, we 
review the execution sequence (without concurrency control) of a request (or a 
request of a transaction) when it is received by the backend. First, the 
directory management process determines the directory attributes for the 
request. This is the attribute search phase. Second, by looking up the 
directory attributes in the AT, directory management determines the descriptor 
id(s) for the request, i.e., the AT contains pointers to the DDIT, which con- 
tains the descriptor ids. This is the descriptor search phase. Using the 
descriptor id(s), directory management then determines the cluster id(s) of 
the cluster (s) that the request needs for execution. This is the cluster 
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search phase. The directory management process performs the address genera- 
tion function and then sends the request to record processing for execution. 

Access to user data is controlled by the database concurrency control 
process (DBCC) , which was presented in [Boyn83] . When the DBCC receives a 
request from directory management, it attempts to lock all of the cluster (s) 
required by the request. Locking clusters involves a series of exercises in 
which access to certain entries of directory data tables is controlled. For 
example, if a cluster number in the CDT is locked, then other requests cannot 
access the Addr field of the CDT. Consequently, the numbered clusters cannot 
be accessed. This is done to insure the consistency of the user data. Before 
we examine why concurrency control is needed in the descriptor search phase, 
we observe that concurrency control is not needed for the attribute search 
phase. Ibis occurs since attributes cannot be added to the database, rather 
they are defined when the database is loaded. However, new attribute values 
may be defined for an attribute if it is a type-C attribute. 

Consider a user database consisting of three attributes, FILE, POPULATION 
and CITY, with the AT, DDIT, and CDT as in Figure 3a, 3b, and 3c, respec- 
tively. Note that FILE and CITY are type-C attributes, and that four type-A 
descriptors are defined for POPULATION. Suppose the name of Cumberland is to 
be changed to Slumberland through the request 

UPDATE ( FILE=Census and CITY=Cumberland) CCITY = Slumberland>. 

Using the AT, directory management determines that the directory attributes 
for the request are FILE and CITY. Now the request enters into the descriptor 
search phase. Using the pointers from the AT, the descriptor search function 
determines that D32 is the descriptor id for (FILE=Census) , and that D21 is 
the descriptor for (CITY=Cumberland) . The insert generated by this update is 

INSERT (<FILE,Census>,<POPULATION,58000>,<CITY,Slumberland>) . 

Since there is no descriptor for <CITY,Slumberland>, a new type-C sub- 
descriptor id, D23, i.e., the id of descriptor 3 for attribute 2, is created 
for the pair <CITY,Slumberland> . 

Now, suppose that a new request, RETRIEVE (CITY NOT= Boston) (CITY), 
arrives at the directory management process for processing. The predicate, 
(CITY NOT= Boston) , specifies the restriction on which records are to be 
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retrieved. The clause, (CITY), specifies which attribute value is to be 
selected. Note that the new request needs all of the descriptors for the 
attribute CITY. Without concurrency control on the descriptor search phase of 
directory management, the following situation could arise. The retrieve 
request could find only D21 and D22 as descriptors for the attribute CITY. 
The update could then take place changing Cumberland to Slumberland causing 
the record to change to a new cluster, C3, defined by the descriptor id set 
{D14,D23,D32} . The retrieve request, however, will only retrieve those 
records of cluster C2, thus missing the newly updated record which also has 
the attribute value of the attribute CITY. Notice that the retrieve should 
not be allowed to do descriptor search until after the new descriptor id D23 
had been created. In general, new type-C sub-descriptors may be generated for 
a type-C attribute, by an INSERT or an UPDATE request. Consequently, we must 
control access to the DDIT by locking the type-C attributes of the AT that 
appear in the request (see Section 2.2). If the type-C attributes of the AT 
are locked, the access of descriptor pointers by later request (s) is prohi- 
bited. Finally, let us examine why concurrency control is needed in the clus- 
ter search phase, by following the INSERT request defined above. 

In the example above, recall that the descriptors defined for the INSERT 
are D32 for FILE, D12 for POPULATION and the newly created D23 for CITY. 
Notice that no cluster is defined in the CDT (Figure 3c) for the set of 
descriptors {D12,D23,D32} . Thus, a new cluster C3 is created for the set of 
descriptors {D12,D23,D32} . The address for C3 is assigned during the address 
generation phase. The record for the insert request, 
(Census, SI umber land, 58000) , is inserted into the secondary storage by record 
processing using the generated address. 

Suppose that we are controlling access at the descriptor search phase. 
When the new request RETRIEVE (CITY N0T= Boston) (CITY) arrives at directory 
management, it must wait until the INSERT finishes descriptor search. When the 
INSERT request releases its lock on the directory attribute CITY, the new 
request locks CITY. Now, when the new request accesses the DDIT it will 
determine that D21, D22, and D23 are the required descriptors. But there is 
still a problem when the new request arrives at the cluster search phase. If 
cluster search for the new request occurs before C3 is created, then Cl and C2 
are determined to be the required clusters. Once again, there will be an 
inconsistency in the data accessed for the RETRIEVE request. Therefore, we 
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must control access to the CDT by locking the descriptors of the DDIT. If a 
request cannot access a given descriptor, it cannot access the cluster ids 
associated with that descriptor. 

Since uncontrolled access of the DDIT and CDT may lead to inconsistency, 
we have developed two concurrency control mechanisms. Descriptor search con- 
currency control (DSCC) controls access to the DDIT by locking directory 
attributes of the AT. Cluster search concurrency control (CSCC) controls 
access to the CDT by locking descriptors of the DDIT. Combining these two 
functions with DBCC yields what was labeled the concurrency control process in 
Figure 2. Lastly, Figure 4 is a pictorial description of how a request moves 
through the process structure of the backends. 

2.2. Concurrency Control in the Descriptor Search ( DSCC ) Phase 

In this section we examine the descriptor search concurrency control 
mechanism. We begin by considering the conditions under which the DDIT 
changes. The DDIT contains for the database three types of descriptors, 
type-A, type-B, and type-C. Recall that type-A and type-B descriptors, and 
type-C sub-descriptors are defined and stored in the DDIT at the database-load 
time. New type-A and type-B descriptors will not be created in the run-time 
environment. However, type-C sub-descriptors are generated and stored in the 
DDIT as new records with new values for type-C attributes are inserted in the 
database. So, we focus on the conditions under which new type-C sub- 
descriptors will be generated. Thus we examine the effect of type-C attri- 
butes appearing in the qualification component of a request. 

2.2.1. Read and Write Control on Attributes 

To control access to the DDIT, we lock the appropriate attributes of the 
AT. The retrieve and delete operations do not modify the DDIT. Retrieve and 
delete requests only read the information in the DDIT. Thus, for a retrieve or 
delete request, the type-C attributes needed by the request are locked for 
read access of the AT. In the previous section, we demonstrated that insert 
and update requests can modify the DDIT. If an insert request is inserting a 
type-C attribute value into the database, and no descriptor exists for that 
value then a new type-C sub-descriptor will be generated. Since there is no 
way to determine if a new descriptor will be generated for an insert request 
until the insert request tries to do descriptor search, the insert request 
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must be granted write control on type-C attributes. So, the type-C attributes 
of an insert request are locked for write access of the AT. 

In the previous section we also saw that the DDIT may be changed by an 
update request if the attribute listed in the modifier is a type-C attribute. 
In this situation, the update must be granted write control on the type-C 
attribute listed in the modifier, implying that the attribute is to be locked 
for write access of the AT. Additionally, an update request is granted read 
control on all type-C attributes listed in the query but not listed in the 
modifier. These attributes are locked as read access of the AT. 

2.2.2. The DSCC Locking Scheme 

The previous two paragraphs have described the standard read/write model 
for concurrent access to records of the database. The read/write model, often 
specified in database textbooks [Ullm82,Date83] , can be characterized in three 
steps. First, multiple read locks on a record are permitted. Second, a write 
lock on a record excludes other reads and writes to that record. And, third, 
a write lock is granted on a record only if the record is not locked (for 
either read or write). In this context, a record is an entry of the directory 
table AT. With respect to the locking scheme of the AT, we conclude that 
type-C attributes of the AT can have either multiple read locks or a single 
write lock. 

2.2.3. The DSCC Data Structures 

We have developed two data structures to store the information needed for 
the descriptor search concurrency control mechanism. The traff ic-unit-to- 
attribute table (TUAT) , is a table internal to the descriptor search con- 
currency control mechanism (DSCC) . The TUAT contains a list of traffic units 
and the type-C attributes needed by each request in each traffic unit. The 
mode of the request, either read or write, is also stored for each type-C 
attribute. This table is used to determine the status of any traffic unit. 
Additionally, this table keeps track of how many requests there are in a tran- 
saction. A sample TUAT table is shown in Figure 5a. The TUAT contains 
entries for four single requests and one transaction of two requests. 

The second data structure, the attribute-to-traff ic-unit table (ATUT) is 
used to keep track of which traffic unit(s) have requested locks on which 
type-C attribute (s) . This table is essentially an inverse of the TUAT. This 
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table contains a queue for each type-C attribute. Each attribute queue con- 
tains an entry for each of the requests requiring that attribute. Each entry 
contains an identifier for the request (the traffic unit and the request 

number) and the mode of access required (read or write) . Figure 5b shows the 

ATUT corresponding to the TUAT shown in Figure 5a. At this point, we can now 
examine the descriptor search concurrency control mechanism. 

2.2.4. The DSCC Lock Conversion Algorithm 

When a traffic unit is ready for descriptor search, directory management 
sends a message to the descriptor search concurrency control mechanism. The 
message consists of a list of all type-C attributes needed by each request of 
the traffic unit, and the type of request, either INSERT, RETRIEVE, UPDATE, or 
DELETE. When such a message is received, DSCC stores the information for the 

traffic unit in the ATUT and TUAT, converting the request type to the 

corresponding mode, either read or write. Then the lock conversion process 
begins. For each attribute needed by each request of the new traffic unit, the 
lock conversion algorithm determines if the lock can be granted. If the lock 
is granted on an attribute, DSCC notifies directory management that the 
descriptor search on that attribute can begin. The process stops when the last 
attribute of the last request has been examined. Directory management noti- 
fies DSCC when the descriptor search on an attribute for a request is com- 
pleted. For insert and update requests, all locks are released at once. For 
non-insert requests, the locks are released one at a time. DSCC then removes 
the attribute from the ATUT and the request from the TUAT, and attempts to 
grant locks for all other request (s) waiting for that attribute. 

Now let's examine the lock conversion function. Suppose that a request R 
needs to lock an attribute A. The queue of the ATUT for attribute A is 
scanned. A pictorial description of the attribute-A queue is given below: 

ATTRIBUTE A : Earlier Requests, R, Later Requests 

There are two cases to consider; whether R needs a read lock or a write lock 
on attribute A. R can obtain a read lock on attribute A if 

(a) R is the first request in the attribute-A queue, 
or 

(b) all earlier requests in the queue have locked 
attribute A for read access. 

R can obtain a write lock on attribute A if and only if R is the first request 
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in the attribute-A queue. (Note: The special case of processing insert (s) gen- 
erated by updates is examined in Chapter 3.) To fully understand the descrip- 
tor search concurrency control mechanism, we step through the algorithm using 
an example. Details of the algorithm are shown in Appendix B. 

Suppose that the new traffic unit is TU5, which consists of two requests 
(see Figure 5b). The first request, Rl, needs a read lock on A2 and a write 
lock on A3. The second request, R2, needs a read lock on A4. Tbe lock 
conversion process first tries to determine if the read lock can be granted on 
A2, the first attribute needed by the first request of the traffic unit. Since 
the two earlier requests, TUI and TU3, both have read locks on A2 (see A2 
queue of ATUT, Figure 5b), the read lock on A2 for Rl of TU5 is granted, i.e., 
an attribute can have multiple read locks. DSCC notifies directory management 
that descriptor search can begin on the attribute A2. Now the algorithm tries 
to lock A3 for write access. Since Rl is not the first request in the ATUT 
queue for A3, the lock is not granted. Now the algorithm begins examining the 
second request. R2 will be granted a read lock on A4 only if all earlier 
requests in the A4 queue of the ATUT table have read locks. Since TU4 is 
requesting a write lock on A4 (see Figure 5b) , the lock is not given to R2. 
Since all the attributes of each request have been examined, the algorithm 
stops. 



2 . 3 ^. Concurrency Control in the Cluster Search ( CSCC ) Phase 

In this section we examine the cluster search concurrency control mechan- 
ism. We begin by considering the conditions under which the CDT changes. An 
entry of the CDT consists of the cluster number, the cluster definition, and 
the secondary storage addresses for the records in the cluster. The cluster 
definition is the set of descriptor ids whose descriptors define the cluster. 
Such a set is called a descriptor-id set . Descriptor-id sets are unique, and 
are used when referring to clusters. They are system data for permanent use. 
On the other hand, the descriptor search phase creates one or more 
descriptor-id groups for a request. A descriptor-id group is a collection of 
descriptor ids which define a set of clusters needed by the request. Thus 
descriptor-id groups are user data for one-time use. Since each cluster is 
defined by a descriptor-id set, we say that a descriptor-id group corresponds 
to the descriptor-id sets defined by the clusters needed by the request. 
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An insert request has exactly one descriptor-id group which corresponds 
to a unique cluster, i.e., a single descriptor-id set. A retrieve, delete, or 
update request can have multiple descriptor- id groups, and each group can 
correspond to multiple clusters (or descriptor-id sets) . We denote 
descriptor- id sets by curly brackets ({...}), and descriptor- id groups by 
square brackets ([...]). 

A new cluster is generated whenever there is a new record whose 
corresponding descriptor-id group is different from all the existing 
descriptor- id sets. Thus, to control access to the CDT, we lock descriptor-id 
groups. If a descriptor- id group is locked, then access to the cluster defin- 
itions is controlled. So, we need to determine what type of access the four 
primary database operations need on descriptor-id groups. 

2.3.1. Read, Insert-Write, and Update-Write Control on Descriptor-Id Groups 

The retrieve and delete operations do not modify the CDT. Retrieve and 
delete requests only read the information in the CDT. Thus, for a retrieve or 
delete request, the descriptor-id groups needed by the request are locked for 
read access. In an earlier section, we showed that an insert request can 
modify the CDT. If the insert request is inserting a record whose 
descriptor-id group does not correspond to an existing descriptor-id set, a 
new cluster will be created. We do not know if a new cluster will be created 
for an insert request until after cluster search, so, the insert request must 
be granted write access on its descriptor-id group. We refer to this as lock- 
ing the descriptor-id group for insert-write control. 

The last type of request, an update request, may also create a new clus- 
ter. In the previous section we presented an example of an update request 
that changed the attribute values in all records of the Census file with city 
equal to Cumberland to Slumberland. In this situation, a new type-C sub- 
descriptor, D24, for (CITY=Slumberland) was created. The descriptor- id group 
generated for the update request in the descriptor search concurrency control 
phase is [D2*,D32]. The descriptor D2* is used to represent all possible 
descriptors for the attribute being modified. Since there is no way to anti- 
cipate the creation of a new type-C sub-descriptor for the update request 
before the record processing phase, we represent all existing and possible 
future descriptors for the attribute city using D2*. Thus, [D2*,D32] 
represents a set of descriptor-id groups. Using this scheme we can logically 
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control access to any request that tries to use a cluster containing a 
descriptor for the attribute city. The descriptor-id group [D2*,D32] is a 
subset of the descriptor-id set {D11,D21,D32} , which defines cluster C2. The 
update request would need write access to the group [D2*,D32], which includes 
cluster C2, in order to prevent other requests from accessing cluster defini- 
tions associated with that group until the update request is completed. This 
prevents other requests from accessing cluster definitions which are supersets 
of [D2*,D32]. Thus, an update request must be granted write control on its 
descriptor-id group(s). We refer to this as locking the descriptor-id 
group(s) for update-write control. 

2.3.2. The CSCC Locking Scheme - The Notion of Conflict-Free 

The differentiation between the insert-write and update-write locks is 
mandated by the complexity of the cluster search locking algorithm. Instead 
of comparing single units, we compare descriptor-id groups. We begin with a 
definition. Two descriptor-id groups are said to be conflict-free if 

(a) both descriptor-id groups require read locks, 
or 

(b) one or both descriptor-id groups require write locks 
and they do not define the same cluster. 

Now let us discuss how to determine if two descriptor-id groups are conflict- 
free. There are two cases to consider depending on whether or not one of the 
requests is an insert. 

Two descriptor- id groups for non-insert requests are conflict-free if 
they contain different descriptors for a common attribute. This occurs since 
a cluster cannot contain two descriptors for an attribute, i.e., it is there- 
fore not possible for the two descriptor-id groups to be subsets of the same 
cluster. As an example, the two descriptor-id groups [D11,D22] and [D11,D23] 
are conflict-free since they have different descriptors for attribute 2, i.e., 
D22 and D23. Conversely, the two descriptor-id groups [D11,D22] and [Dll] are 
in conflict since [Dll] is contained in [D11,D22] and therefore the groups can 
be in the same cluster. Further, observe that [Dll] and [D22] are also in 
conflict, since there may be a cluster containing them both. 

If one or both of the descriptor-id groups represents an insert request, 
the test for conflict-free is different since the descriptor-id group for an 
insert request represents a unique cluster. If both requests are inserts. 
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then the descriptor- id groups are conflict-free if the descriptor-id groups 
are not identical. If one of the requests is a non-insert request, then the 
descriptor- id groups are conflict-free if the descriptor-id group for the 
non-insert request is not contained in the descriptor-id group for the insert 
request. 



2.3.3. Two Categories of Locks 

To keep track of which descriptor-id groups have obtained either a read, 
insert— write, or update— write lock, we introduce two categories of locks on 
descriptor-id groups: "to -be-used " and " being-used ". As soon as a request 
reaches a backend, it locks the descriptor-id group(s) it needs in the "to- 
be-used" category. The "to-be-used" category of locks secures the request's 
claim for a "being-used" lock on a descriptor-id group. In this way, we can 
prevent a later request from locking a descriptor-id group for which an ear- 
lier request is waiting. Before the request can do cluster search, the locks 
on all descriptor-id group (s) must be converted to the "being-used" category. 
The "being-used" lock signifies that a request has access to a descriptor-id 
group. A "being-used" lock is granted on a descriptor-id group if that group 
is conflict-free with all earlier descriptor-id group(s) . 

2.3.4. The CSCC Data Structure 

To store the information needed by the cluster search concurrency control 
mechanism, the traff ic-unit-to-descriptor-id-groups table (TUDIGT) was 
developed. The TUDIGT contains a list of traffic units and the descriptor-id 
groups needed by each request in each traffic unit. The mode, either read, 
insert-write, or update-write, and the category, either "to-be-used" or 
"being-used", of each descriptor-id group is also stored. Figure 6 shows a 
sample TUDIGT which contains entries for four requests and one transaction of 
two requests. We now examine the cluster search concurrency control mechanism. 

2.3.5. The CSCC Lock Conversion Algorithm 

When a traffic unit is ready for cluster search, directory management 
sends a message to the cluster search concurrency control mechanism. The mes- 
sage consists of a list of all descriptor-id groups needed by each request of 
the traffic unit, and the type of request, either INSERT, RETRIEVE, UPDATE, or 
DELETE. The information for the new traffic unit is stored in the TUDIGT, 
with the request type converted to the appropriate mode, i.e., either read. 
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Figure 6 . A Sample Traf f ic-Unit-To-Descriptor-Id-Groups Table (TUDIGT) 
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insert-write, or update-write. After the information for the new traffic unit 
has been stored, the lock conversion process begins. For each descriptor-id 
group needed by each request of the new traffic unit, the lock conversion 
algorithm determines if the lock can be granted. A lock on a descriptor-id 
group can be granted if that group is conflict-free with all earlier 
descriptor- id groups. If all locks for all descriptor-id groups for a given 
request are converted to "being-used", CSCC notifies directory management to 
begin cluster search. The process stops when the last descriptor-id group of 
the last request has been examined. Directory management notifies CSCC when 
the cluster search for a request has completed. CSCC removes the information 
for the request from the TUDIGT and attempts to grant locks for all other 
waiting request (s). 

Once again, we step through the algorithm using an example. Suppose that 
the new traffic unit is TU4, which consists of one request (see Figure 6) . We 
will assume that the information for TU5 has not been received by CSCC. The 
first descriptor-id group [D11,D24], needs an update-write lock. We compare 
[D11,D24] with all earlier descriptor-id groups. [D11,D24] is conflict-free 
with [D12,D22] since Dll and D12, descriptors for attribute 1, are different. 
[Dll ,D24] is conflict-free with [D11,D21], [D11,D22] and [D23] since the 
descriptor for attribute 2, D24, is different from D21, D22, and D23. Thus, 
the "being-used" lock is granted since [D11,D24] is conflict-free with all 
earlier requests. Now the algorithm tries to obtain an update-write lock on 
the second descriptor-id group, [D12,D22]. While [D12,D22] is conflict-free 
with [Dll ,D21] of TUI, [D12,D22] conflicts with [D12,D22] of TUI. Therefore, 
the lock is not granted. Since all descriptor-id groups for the request have 
been examined, the algorithm stops. 

There are two final notes on the cluster search concurrency control 
mechanism. First, the detailed design of CSCC can be found in Appendix B. 
Second, the special case required to handle insert (s) caused by an update 
request is examined in Chapter 3. 
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3. THE REQUEST EXECUTION OF AN UPDATE REQUEST 



In this chapter we examine the execution sequence of an update request. 
An update request modifies records of the user database. Under normal cir- 
cumstances, an update request will retrieve a record from the user database, 
update the specified record value, and write the record back to the secondary 
storage. The update request will continue until all appropriate records have 
been modified. However, under some conditions the update of a record is logi- 
cally equivalent to the retrieval of the record, the deletion of the record, 
the creation of a new record, and the insertion of the new record. In such a 
situation, an insert request must be generated. Thus, processing an update 
request must proceed in two logical phases. First, all the records to be 
updated are retrieved and either modified or converted to insert requests. 
This is the record-modification phase of the update request. Second, the 
insert requests are performed. This is the gene rated- insert phase of the 
update request. To begin, we focus on the conditions under which an update 
request generates an insert request. 

Suppose that a user wants to increase all populations between 10,000 and 
40,000 people by 15,000 people m the Census file. The update for this 
request is listed below: 

UPDATE ((FILE = Census) and 

(POPULATION >= 10000 and POPULATION <= 40000) ) 

POPULATION = POPULATION + 15000 > 

In referring to Figure 3b, the descriptors needed by the request are D31 for 
the clause (FILE = Census) and Dll for the clause (POPULATION >= 10000 and 
POPULATION <= 40000) . The cluster corresponding to these descriptors would be 
Cl, which is defined by the descriptor-id set {Dll ,D21 ,D31} . There are two 
cases to consider. First, suppose that there is a record in the user database 
with population 12,000. The record will be modified by the update request, 
changing the record value for population to 27,000 (i.e., 12,000 + 15,000). 
The descriptor id for this record value is still Dll, so the modified record 
is written back to the secondary storage. 

Now, assume that there is a record in the user database with population 
37,000. This record will be modified by the update request, changing the 
record value for population to 52,000 (i.e., 37,000 + 15,000). The descriptor 
id for this record is now D12, 50001 <= POPULATION <= 100000. But notice that 
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there is not a cluster defined for this record (see Figure 3c) . Thus a new 
cluster id corresponding to the descriptor ids, D12, D21 and D31, would be 
entered into the COT. When a record moves from one cluster to another cluster 
(either an existing cluster or a new cluster) , as a result of the update 
action, we say that the record has changed cluster . In this example, the 
record must change into a new cluster which results in the definition of a new 
cluster. However, the fact that a new cluster is created is not important. 
The key point is that the record changed cluster. Since the record has 
changed cluster, it cannot simply be written back to the secondary storage 
with the modified record value. Instead, the old record will be marked for 
deletion and an insert request for the new record will be generated. (Note: 
In this example a new cluster is defined from existing descriptor ids.) 

In general, an update request will generate an insert request if the 
record being modified changed cluster. The insert request generated as a 
result of the update request is characterized as a generated-insert request. 
In the example above, the generated-insert request would be: 

INSERT (<FILE = Census>, POPULATION = 52000>,<CITY = Cumberland)*) 

The record to be inserted contains the modified value of the population attri- 
bute along with the values of city and file stored in this record. The record 
values. Census for file (descriptor D32) , 52,000 for population (descriptor 
D12) and Cumberland for city (descriptor D21) define the cluster {D12,D21 ,032} 
for this record. Thus, an update request can consist of the two logical 
phases specified above. The remainder of this chapter examines the two logical 
phases, and how generated-insert requests are processed by the descriptor 
search, cluster search and database concurrency control mechanisms. 

3^. The Two Phases of an Update Request 

In this section we examine the two phases of an update request, the 
record-modification phase and the generated-insert phase. It would be desir- 
able to overlap these two phases. Otherwise the insert requests must be stored 
for later processing. Records may be inserted into a cluster as soon as the 
record-modification phase for that cluster has completed. The rest of this 
section is divided into two parts. First we examine the execution sequence 
for an update request and its generated-insert requests. This analysis does 
not assume any overlap between the record— modification and generated-insert 
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phases. Second, we examine now the overlap of the two phases can be imple- 
mented . 

To motivate this discussion, we recap the execution sequence of a traffic 
unit (consisting of one or more requests) . The traffic unit is received by 
the controller from the host. After processing the traffic unit in the con- 
troller, request preparation sends the traffic unit to the directory manage- 
ment process of each backend for execution, Each request in the traffic unit 
moves through the backend (see Figure 4) finally reaching the record process- 
ing process. Record processing manages the physical data operations, i.e., 
inserting a new record for an insert request, retrieving records for a 
retrieve request, etc. When a request has finished processing, record pro- 
cessing sends the results to the post processing function of the controller. 
Post processing collects the results for a traffic unit and forwards them to 
the host. 

3.1.1. Execution of an Updat .quest without Overlap 

We begin by examining how an update request is processed by record pro- 
cessing. We are assuming that there is no overlap of the record-modification 
and generated-insert phases. After the database concurrency control mechanism 
determines an update request can execute, directory management generates the 
cluster addresses needed by the update request. Record processing cycles 
through the clusters track by track, examining all records which satisfy the 
update request, i.e., the records which satisfy the query component of the 
update request. After retrieving a record, and determining that the record 
satisfies the query, record processing recalculates the specified record value 
using the modifier. Then, record orocessing sends the attribute being modi- 
fied, along with the old and new record values for this attribute, to direc- 
tory management to determine if the record has changed cluster. 

There are two cases to consider. If the updated record has not changed 
cluster, the modified record written back to the secondary storage. If the 
modified record changed cluster, the old record is marked for deletion, and 
the modified record is sent by record processing to request preparation (of 
the controller), i.e., the "changed-cluster-record" message in Figure 7, so 
that an insert request f or that record can be generated. Now we follow the 
actions taken by a generated-insert request. 
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After request preparation receives the "changed-cluster-record" message 
(see Figure 7) from record processing, it generates an insert request. The 
generated-insert request is broadcast to the directory management process of 
every backend (the "generated-insert-request" message from request preparation 
to directory management in Figure 7) . This request is the same as an insert 
request, except that it is marked so that it may be associated with the update 
request that caused it. The generated-insert request flows through the pro- 
cess structure of the backend (see Figure 4). However, since this insert is 
associated with a unique update request, the actions of the generated-insert 
request in the descriptor search, cluster search, and database concurrency 
control mechanisms must be processed as a special case. 

The last stages of the record-modification phase in processing an update 
request occur when all records satisfying the query have been examined. The 
record processing process sends a message informing request preparation that 
there will be "no-more-changed-cluster-records" generated (see Figure 7) . 
Request preparation keeps track of the generated-insert requests since an 
insert request may be generated by any backend, and an insert request gen- 
erated at one backend may be carried out at another backend. When request 
preparation has received the "no-mo re-changed-cluster-records" message from 
every backend, the record-modification phase of the update request has fin- 
ished. Request preparation notifies the directory management process of every 
backend that there will be "no-more-generated-insert-requests" (see Figure 7) . 
When the directory management process of a backend has sent all generated- 
insert requests to record processing, and has also received the "no-more- 
generated- insert-requests" message from request preparation, directory manage- 
ment sends a "no-more-generated-insert-requests" message (see Figure 7) to 
record processing. Finally, when record processing has finished executing all 
of the generated-insert requests and has received the "no-more-generated- 
insert-requests" message from directory management, it sends an "update-done" 
message (see Figure 7) to directory management. Directory management releases 
space being used by the update request, and then notifies concurrency control 
(with the "update-done" message) that it can release locks being held by the 
update request. 
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3.1.2. Execution of an Update Request with Overlap 

When deciding how to process a generated-insert request, we were faced 
with the problem of when to store the new records created by the generated- 
insert requests. These records must wait until the record-modification phase 
has finished. If the record for a generated-insert request is allowed to be 
placed in the secondary storage by record processing before the record- 
modification phase has finished, the record-modification phase may modify the 
newly inserted record. Such a modification would result in an inconsistent 
user database. We have two places where the records created by the 
generated-insert requests can be temporarily stored, in either directory 
management or record processing. In chosing to store them in record process- 
ing, we permit the records to progress through the system as far as possible. 

We also discovered that we could easily increase the "intelligence" of 
the record processing process. Recall that when processing an update request, 
record processing examines the records track by track. When record processing 
has finished examining a track, insertion of records of generated-insert 
requests into that track can begin. Such a generated-insert request will not 
have to wait until the record-modification phase of the update request is done 
before placing new records on the secondary storage. Thus the retrieval and 
generated-insert phases can be successfully overlapped. 

In the current version of IHDBS we have yet to implement procedures to 
handle the overlap of the two phases. However, we have developed our design 
so that the eventual processing of the record-modification and generated- 
insert phases concurrently can be implemented without any major modifications. 
Lastly, we examine the steps required to process generated-insert requests. 

When database concurrency control determines that an insert (any insert) 
request can execute, directory management generates an address for the 
request. Directory management then forwards the insert request and address to 
record processing. Record processing checks to see if the insert request is a 
generated-insert request. If it is, record processing will hold the insert 
request until the record-modification phase of the associated update request 
has completed. Otherwise, record processing executes the insert request. 



- 28 - 



2*2. • Concurrency Control for Generated-Insert Requests 



In this section we examine the steps taken to process a generated-insert 
request in the descriptor search, cluster search, and database concurrency 
control mechanisms. This section is divided into two parts. First, we con- 
sider the various design issues when developing a strategy to process 
generated-insert requests. These issues are reviewed for the three con- 
currency control mechanisms. Second, we examine the implementation details 
for processing generated-insert requests in the three concurrency control 
mechanisms. 

3.2.1. The Design Issues 

We begin this section by presenting the general idea involved in the pro- 
cessing of a generated-insert request. A generated-insert request is caused 
by a particular update request. The update request has locks on all attri- 
butes, descriptor-id groups, and clusters needed by the generated-insert 
request. When the generated-insert requests of an update request are attempt- 
ing to secure locks (on attributes, descriptor-id groups, or clusters) , they 
have a logical priority over other later requests trying to obtain the same 
locks. This is due to the fact that the generated-insert requests represent 
the second phase in the execution of an update request, i.e., the generated- 
insert requests are part of the update request. 

However, we still must be careful when processing generated-insert 
requests of the update request. In the descriptor search concurrency control 
mechanism, generated-insert requests cannot execute concurrently since they 
may conflict with each other, e.g., two generated-insert requests may both 
cause the generation of the same new descriptor id. In the cluster search con- 
currency control mechanisms, generated-insert requests are also prohibited 
from concurrent execution, since they may create new clusters. However, in 
the database concurrency control mechanism, generated-insert requests of an 
update request are allowed to execute concurrently. This occurs since two or 
more generated-insert requests are always compatible, i.e., the consistency of 
the user database will not be affected if the generated-insert requests are 
executed concurrently. The remainder of this section analyzes the design 
issues for processing generated-insert requests in the three concurrency con- 
trol mechanisms, using an example which is developed in the next subsection. 



29 



(A) On Descriptor Search Concurrency Control 

Suppose that we are processing the update presented in Section 2.1, 

UPDATE ( (FILE=Census) AND (CITY=Cumberland) ) <CITY=Slumberland> . 

This update changes the attribute values in all records of the Census file 
with city Cumberland to Slumberland. When the update request is processed, 
the descriptor search concurrency control mechanism will lock the attributes 
FILE and CITY of the ATUT. FILE is locked for read access and CITY is locked 
for write access by the update request. When the update request finally 
reaches record processing, records for the appropriate cluster (s) are 
retrieved. When a record containing the values (Census, Cumberland) is found, 
the CITY record value is changed to Slumberland. Since this record changed 
cluster, a generated-insert request is created, say, 

INSERTJL (<FI LE,Census>,<CITY, Slumber land>, POPULATION, record_value_l>) , 

where record_value_l is the record value for POPULATION. To fully illustrate 
the problem, let us suppose that a second generated-insert request was 
created, say, 

INSERT_2 (<FI LE ,Census> , <CITY , Slumber land> , POPULATION , record_val ue_2>) , 

where record_value_2 is another record value for POPULATION. Both of these 
requests will have been sent to descriptor search concurrency control. 

The generated-insert request, INSERT_1, will be processed first in 
descriptor search concurrency control. Assume that INSERT_1 will be granted 
locks on the attributes FILE, CITY, and POPULATION. DSCC will notify direc- 
tory management that it may do descriptor search for INSERT_1. At this point, 
since there is no descriptor id for (CITY=Slumberland) , a new type-C sub- 
descriptor, with id D23, will be defined. Notice that INSERT_2 also needs 
access to this new descriptor, so it must wait until INSERT_1 has finished 
descriptor search before it can lock CITY. Thus, these two generated-insert 
requests cannot be executed concurrently. 

Lastly, it is important to examine the types of locks given to INSERT 1 
on the attributes FILE, CITY and POPULATION. In Section 2.2.1 we concluded 
that an insert request needs write access on all attributes needed by the 
request. However, a generated-insert request is a special type of insert 
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request. The update request is holding locks on the attributes needed by the 
generated-insert request. One of the attributes, CITY (the attribute being 
modified) , is being held as write access. FILE is being held by the update 
request for read access. Since there is no way that the generated-insert 
request INSERT_1 can change the value of FILE, i.e., create a new descriptor, 
it is only necessary that this insert request has read access on FILE. Addi- 
tionally, we know that the update request does not hold a lock on POPULATION. 
Once again there is no way that the generated-insert request can create a new 
value for POPULATION. In this situation, since the update is not locking 
POPULATION, we ignore the attribute, and simply notify directory management 
that descriptor search for the request can commence. Now let us see whether 
or not there is any problem in the cluster search concurrency control mechan- 
ism. 



(B) On Cluster Search Concurrency Control 

We first assume that the record values for POPULATION, record_value_l and 
record_value_2, are represented by the same descriptor, say Dll. Working with 
the two generated-insert requests INSERT_1 and INSERT_2, we find that INSERT_1 
and INSERT_2 will have the descriptor-id group, [D11,D23,D321 , after the 
descriptor search phase. Recall that for insert requests, a descriptor-id 
group defines a unique cluster. Since their descriptor-id groups are identi- 
cal, both requests, INSERT_1 and INSERT_2, will define the same cluster. 
Since the cluster {Dll ,D23,D32} does not yet exist, only one request, 
INSERT_1, can be passed to directory management for cluster search. INSERT_2 
must wait since a new cluster is to be defined. Once again, we cannot allow 
concurrent execution of generated-insert requests. 



(C) On Database Concurrency Control 

Finally, we arrive at the database concurrency control mechanism. At 
this point, we assume that a new cluster, C5, corresponding to {D11,D23,D32} 
has been created. The first thing we observe is that the update request is 
only locking the cluster Cl, since C5 did not exist at the time the update 
request locked Cl. When the first generated-insert request arrives, it asks 
for C5. We first lock C5 for the update request. Both INSERT_1 and INSERT_2 
are requesting locks on C5. Since insert requests are compatible, the locks 
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for both requests are also granted. The requests can now be executed con- 
currently, since the records for INSERT_1 and INSERT_2 will be inserted into 
different blocks of the same or different track. 



(D) Conclusions on Generated-Insert Requests 

We must take special care when processing generated-insert requests. We 
have a special two-step procedure when handling generated-insert requests. 
First, we must include the generated-insert request as part of the original 
traffic unit, i.e., the traffic unit which contains the update request that 
caused the generated-insert request. Such a step is necessary since the 
generated-insert request is the second phase of the update request. Second, 
depending upon the concurrency control mechanism, we have different degrees of 
concurrency for the generated-insert requests. In the descriptor search con- 
currency control mechanism only one request may write to the DDIT. In the 
cluster search concurrency control mechanism, only one generated-insert 
request may write to the COT. In the database concurrency control mechanism, 
we permit multiple writes to the user database. 

3.2.2. The Implementation Details 

In this section we investigate the two-step process of handling 
generated-insert requests in the DSCC, CSCC, and DBCC respectively. The two 
steps are, one, entering the information for a generated-insert request into 
the concurrency control data structures, and two, assigning locks for a 
generated-insert request. 



(A) Processing Generated-insert Requests in DSCC 

In this section we examine the steps taken to process a generated-insert 
request in the descriptor search concurrency control mechanism. There are two 
main differences when processing a generated-insert request. First, the 
information for the generated-insert request must be entered into the 'IUAT and 
ATUT data structures in the designated place. Second, the locking scheme for 
a generated-insert request varies slightly from the one described in Section 
2.2.2. We begin by considering how the information for the generated-insert 
request is entered into the TUAT and ATUT. 
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The generated-insert request is received by the descriptor search con- 
currency control mechanism with a list of the type-C attributes needed by the 
request and a code associating the request with a unique traffic unit and the 
update request within the traffic unit that caused the generated-insert 
request. The list of type-C attributes needed by the first generated-insert 
request (of a particular update request) is entered into the TUAT after the 
corresponding update request and before any later requests of the traffic 
unit. Entries for subsequent generated-insert requests are inserted after 
earlier generated-insert requests and before any later requests of the traffic 
unit. 



In general, an insert request must lock all its attributes for write 
access. However this is not the case for a generated-insert request. As an 
example, suppose that the generated-insert request was caused by the first 
request in traffic unit TU3. The portion of the TUAT table showing TU3 is 
reproduced below. 



Traffic-Units | | 


I Requests 


TU3 1 

II 


A2 A3 I A4 

1 w r | r 



-H + + 

Further suppose that the generated-insert request needs write access on attri- 
butes A2, A3, and A4. However, since the update request is locking attribute 
A3 for read access, attribute A3 for the generated-insert request only 
requires read access, i.e., there is no way that a new descriptor can be 
created on attribute A3. Thus descriptor search concurrency control will 
change the write access to read access before entering A3 into the TUAT. When 
the information for the generated-insert request is inserted into the TUAT, 
the TU3 queue now appears as: 



Traffic-Units | | 

_ 


Requests 






TU3 j 


A2 A3 


1 A2 A3 


* A4 “t” 
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w r 
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w | 
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The generated-insert request must be placed after the update request (and any 
other earlier generated-insert requests for this update request) and before 
any later requests, since the generated-insert request is part of the update 
request, and must be processed after any earlier generated-insert requests and 
before any later requests. This is done to insure the consistency of the 
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directory information, specifically the DDIT data structure. 

The information for the generated-insert request must also be entered 
into the ATUT. For each type-C attribute needed by the generated-insert 
request, an entry consisting of the traffic unit number, the request number, 
and the mode of access (either read or write) , is created. Each of these 
entries is inserted into the corresponding attribute queue of the ATUT in the 
following manner. If the update request request also needed this attribute, 
the entry is inserted into the ATUT after the entry for the update request 
(and any other entries for earlier generated-insert requests for this update 
request) and before entries for any later requests of the same or different 
traffic unit. If the update request did not need this attribute, the entry is 
discarded. 

Following through with the example above, the portion of the ATUT before 
the entries for the generated-insert request are added is: 



Type-C 
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After the entries for the generated-insert are added, the ATUT table is 



Type-C 1 
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where RG denotes the generated-insert request. The RG entry for attribute A4 
was discarded, since the original update request didn't lock A4. After enter- 
ing the generated-insert request into the TUAT and ATUT, the processing of the 
request begins. The processing of a generated-insert request is the same as 
any other request (see Section 2.2.4) except for the lock conversion function. 
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Suppose that the generated- insert request RG needs to lock attribute A. 
The queue of the ATUT for attribute A is scanned (shown below) . 

ATTRIBUTE A : Earlier Requests, RG, Later Requests 
There are two cases to consider: 

(a) when the earlier request adjacent to RG 
is the update request that caused it, 

or 

(b) when the earlier request adjacent to RG 
is another generated-insert request. 

In case (a) we have two possibilities, whether RG is requesting a read or 
write lock on attribute A. For either possibility, since RG is the first 
insert generated by the update request, the lock, either read or write, is 
granted, regardless of the type of lock being held by the update request. In 
case (b) , the lock is not granted since a new descriptor may be created by the 
earlier generated-insert request currently holding the lock. The net effect 
of this locking scheme is the serialization of the generated-insert requests 
of an update request. In the example given above, the generated-insert 
request will be granted a write lock on A2 and a read lock on A3. 

At first glance, there seems to be a serious problem with case (a). When 
the update request adjacent to the generated-insert request has a write lock, 
we are also granting the generated-insert request a write lock. This seems to 
contradict the standard read/write model described in Section 2.2.2. There 
are two key observations to make here. First, we are only using the update 
request to retain the lock on a specific attribute. Second, for a generated- 
insert request to arrive at DSCC, the update request must be currently in 
record processing, i.e., finished with descriptor search. The generated-insert 
request is logically part of the update request, and must be allowed to per- 
form descriptor search before any later requests. Also, since the update 
request has finished descriptor search, there is no chance that inconsistency 
may develop in the DDIT. Therefore, we use the update request to hold the 
lock for any of its generated-insert requests. 



(B) Processing Generated-insert Requests in CSCC 

This section examines the steps taken to process a generated-insert 
request in the cluster search concurrency control mechanism. Once again, 
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there are two main differences when processing a generated-insert request; 
entering the generated-insert request into the TUDIGT (see Figure 6) and pro- 
cessing the generated-insert request through the lock conversion scheme. 

The generated-insert request is sent to cluster search concurrency con- 
trol with the descriptor-id group needed by the request, the mode of the 
request (insert-write) , and a code associating the request with a unique 
traffic unit and the update request within that traffic unit that caused the 
generated-insert request. The descriptor-id group for the first generated- 
insert request (of a particular update request) is entered into the TUDIGT 
after the corresponding update request and before any later requests in the 
traffic unit. Descriptor-id groups for subsequent generated-insert requests 
are inserted after earlier generated-insert requests for that update request 
and before any later requests in the traffic unit. The situation here is the 
same as described in the previous section for the TUAT table. After entering 
the generated-insert request into the TUDIGT, the processing of the request 
begins. The processing of a generated-insert request is the same as any other 
request (see Section 2.3.5) except for the lock conversion scheme. 

The locking scheme is somewhat simplified for a generated-insert request. 
As described in the previous section, we are just using the up>date request to 
hold the lock on its descriptor-id group for any generated-insert requests. 
Also, recall that an update locks its descriptor-id group(s) for update-write 
access (see Section 2,3.1). The generated-insert request will be granted an 
insert-write lock on its descriptor-id group only if it is adjacent to the 
update request that caused it. Other later generated-insert requests will not 
be granted an insert-write lock, since their descriptor-id groups will con- 
flict with the generated-insert request currently holding an insert-write 
lock. In fact, the descriptor-id group for the generated-insert request hold- 
ing the insert-write lock will conflict with any other generated-insert 
request's descriptor-id group on the attribute being modified, i.e., the two 
groups will not be conflict-free. Once again, we are guaranteeing the seriali- 
zation of the generated-insert requests. 



(C) Processing Generated-insert Requests in Database Concurrency Control 

This section examines the steps taken to process a generated-insert 
request in the database concurrency control mechanism. Once more, there are 



- 36 - 



two main differences when processing a generated-insert request; entering the 
information into the traff ic-unit-to-cluster table (TUCT) and cluster-to- 
traffic-unit table (CTUT) (see [Boyn83] for a description of these data struc- 
tures and an explanation of the database concurrency control algorithm) , and 
processing the generated-insert request through the lock conversion function. 

The generated-insert request is sent to database concurrency control with 
a cluster needed by the request, and a code associating the request with a 
unique traffic unit and the update request within that traffic unit that 
caused the generated-insert request. An entry of the TUCT for a generated- 
insert request consists of the cluster needed by the request, the type of the 
request (an insert), and the category of lock being held ("to-be-used"). The 
entry for the first generated-insert request (of a particular update request) 
is inserted into the TUCT after the corresponding update request and before 
any later requests in the traffic unit. Entries for subsequent generated- 
insert requests are entered into the TUCT after earlier generated-insert 
requests for that update request and before any later requests in the traffic 
unit. 



The information for the generated-insert request is also entered into the 
CTUT. For the cluster needed by the generated-insert request, an entry con- 
sisting of the traffic unit number, the request number, the type of request 
(insert), and the category of lock being held ("to-be-used"), is inserted into 
the corresponding cluster queue of the CTUT. If the update request which 
caused the generated-insert request also needed this cluster, this entry is 
inserted into the CTUT data structure after the entry for the update request 
that caused it (and any other earlier generated-insert requests for this 
update request) and before entries for any later requests of the same or dif- 
ferent traffic unit. If the update request did not need this cluster, i.e., a 
new cluster was defined, then an entry for the update request is created 
(locking the cluster as "being-used"), and entered into the cluster queue. 
The entries for generated-insert requests are entered at the end of this clus- 
ter queue. The processing of a generated-insert request in the database con- 
currency control mechanism is the same as any other request (see [Boyn83] ) 
except for the lock conversion scheme. 

Briefly, the locking scheme for the database concurrency control mechan- 
ism tries to convert locks on clusters needed by a request from "to-be-used" 
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to "being-used". If a "being-used" lock is not granted, a "waiting" lock is 
assigned to the request for that cluster. The "waiting" lock secures the 
request's claim for a "being-used" lock on a cluster. If all locks on clus- 
ters needed by a request are converted to "being-used", the request is passed 
to directory management. Directory management does the address generation for 
the request and forwards the request and generated address (es) to record pro- 
cessing. Generated-insert requests are compatible. Since the update request 
has secured the lock on a cluster, a generated-insert request is given a 
"being-used" lock on the cluster that it needs. Thus, the locking scheme is 
very straight-forward. 
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4. THE SECONDARY-MEMORY-BASED DIRECTORY MANAGEMENT 



In this chapter we describe the implementation of directory management 
using the secondary storage. Let us first recall the main functions of direc- 
tory management. Directory management receives traffic units from the con- 
troller. Directory management processes the traffic unit one request at a 
time. Each request passes through a number of phases under the control of the 
directory management process. These phases are: attribute search, descriptor 
search, cluster search, and address generation (see Figure 4 again) . To 
proceed through these phases, directory management accesses the directory 
data, i.e., the attribute table (AT), the descriptor-to-descriptor-id table 
(DDIT) , and the cluster-definition table (CDT) . 

Version A through version E stored the directory data in the primary 
memory. In the final version, version F, the directory data is stored in the 
secondary storage. When the directory data is in the secondary storage, pro- 
cessing is more complex because there is a delay every time some directory 
data is to be read from or written to the secondary storage. Additionally, 
when a new type-C sub-descriptor is created, a new cluster is defined, or an 
address of a new record is allocated, the insertion of new directory data into 
the tables maintained on the secondary storage must be performed. Thus, in 
the following sections we describe the processing required for each phase of 
the secondary-memory-based directory management, attribute search, descriptor 
search, cluster search and address generation. Appendix D contains an 
analysis of the algorithms required for the insertion of new directory data. 
To simplify the discussion, we introduce some new notation and concepts. 

In the attribute search phase, we process the query component of a 
request one predicate at a time. For each predicate, we must determine the 
attribute-id for the attribute in that predicate. In the descriptor search 
phase, we also process a request one predicate at a time. For each predicate, 
we determine the corresponding descriptor ids. We then create the Cartesian 
product of the descriptor ids of each predicate. Each result of that Carte- 
sian product is a descriptor-id group. In the cluster search phase, we pro- 
cess a request using descriptor-id groups. For each descriptor-id group, we 
determine the corresponding cluster ids. Finally, in the address generation 
phase, we process a request using cluster ids. For each cluster id, we deter- 
mine the corresponding secondary storage addresses of the records in that 
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cl uster . 



The processing during each phase of directory management is described in 
terms of the state and state-transition . Each state is represented by a rec- 
tangular box, which contains a description of the actions which take place in 
the state. The description of each phase will be given using a state transi- 
tion diagram. These states include reading a particular type of data such as 
an attribute table node or waiting for concurrency control to grant a needed 
lock. 



Lastly, to further simplify the discussion, we will not mention the wait- 
ing state. The waiting state occurs when an area of the primary memory, 
referred to as a buffer, is required for an I/O operation. Since there are 
only a finite number of buffers in the system, the waiting state is entered 
whenever a buffer is not available for the I/O operation. 

4^. The Attribute Search 

The first phase of directory management is the attribute search. In this 
phase the attribute-id, if any, for the attribute in each predicate of the 
query is determined, as well as a pointer to the location of the descriptors 
in the DDIT for that attribute. 

As described in [Boyn83] , the attribute table is stored in a B-tree. A 
sample B-tree is in Figure 8. Processing is performed for each predicate in 
the query. Each node of the B-tree is stored in a different secondary storage 
location. Therefore, the nodes must be read and processed one at a time until 
the attribute is found. In addition, before the descriptor search can begin 
on a type-C attribute, that attribute must be locked by concurrency control 
(see Chapter 2 again) . 

The attribute search is described in more detail in Figure 9. Predicates 
are processed one at a time. For each predicate in the query the processing 
is as follows. First, the root node of the AT is read (marked with the number 
1 in Figure 9) . Then a sequence of nodes of the AT must be read; either the 
attribute is found or a leaf node is reached without finding the attribute 
(marked 2 in Figure 9) . When the attribute is not in the AT, then we assume it 
is a non-directory attribute. In this case, no descriptor search is needed 
for that attribute, so the descriptor search for that predicate is finished by 
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Figure 9. The Attribute Search for a Predicate in a Request 



default (3). When the attribute is found in the AT, there are two possibili- 
ties. If the attribute is not a type-C attribute, then descriptor search can 
begin(4). However, if the predicate we are processing contains a type-C 
attribute, the attribute must be locked (5) before descriptor search can begin. 
In either case, when the attribute is found, the attribute id and the pointer 
to the descriptors for the attribute can now be obtained from the AT and made 
available to the next phase of directory management. We say that the attri- 
bute search is done for the attribute (6) and the descriptor search begins for 
the same attribute (see Figure 11) . 

The previous discussion focuses on the processing of one predicate of the 
query component. The attribute search phase processes all predicates of the 
query component, before the descriptor search phase for that request can 
begin. Thus, we can have an extra looping structure superimposed on the state 
diagram of Figure 9, which cycles through all predicates of the query com- 
ponent for a given request. 

_4 .2. The Descriptor Search 

The second phase of directory management is the descriptor search. In 
this phase the descriptor-ids corresponding to the predicate are determined. 
These descriptor-ids are stored in a B+tree as shown in the sample 
descriptor-to-descriptor-id table (DDIT) in Figure 10. Briefly, the DDIT con- 
sists of index nodes and sequence nodes. Index nodes are used to traverse the 
B+tree. Sequence nodes contain the information for a particular descriptor, 
e.g., the descriptor id, and the range of values for that id. Depending on 
the relational operator involved, the descriptor search first must determine 
either the leftmost sequence node, (for the operators, <, <=, NOT=) , or an 
intermediate sequence node (for the operators, >,>=,=). If a range of 
values is required, then the descriptor search must follow the sequence nodes 
to determine the other descriptors. For an illustration, let us refer to Fig- 
ure 10 and look at two examples. For the predicate AGE < 30, the descriptors 
Dl, D2 and D3 must be determined. This is done by first retrieving the begin- 
ning sequence node, the one containing Dl and D2. Then the second sequence 
node must be examined to find D3. For the predicate 32 < AGE < 39, the 
descriptor corresponding to AGE = 32 must be determined. Ibis descriptor is 
D4, which is in the second sequence node. Then D5 can be determined from the 
third sequence node. 
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The steps of descriptor search are shown in Figure 11. First the root 
node is read and processed (1) . If there is only a root node for this attri- 
bute, i.e., the root node is a sequence node , then processing is finished (2). 
If the root node is not a sequence node, then the appropriate initial sequence 
node must be found. This will be the leftmost sequence node if the predicate 
relation is <, <= or NOT=. Otherwise, an intermediate node must be found. 

In the first case the search is done by reading the leftmost child of the 
root node (3) and then continuing to read the leftmost child down the tree 
until the leftmost sequence node is found (4). If no additional sequence nodes 
are required, then the descriptor search is finished for this predicate (5) . 
If additional sequence nodes are required, one(6) and possibly several (7) more 
sequence nodes are read. In the second case, i.e., an intermediate node must 
be found, a search down the B+tree is required (8 ,9) . After the sequence node 
is found and if no additional sequence nodes are needed, the descriptor search 
is finished (10) . Otherwise, additional sequence nodes are still required. 
One (11) and possibly several (7) more sequence nodes are read. After all the 
required sequence nodes have been read, the descriptor search is finished (12) . 
At the end of this phase, the descriptor ids of the descriptors corresponding 
to the given predicate are found and made available to the next phase, the 
cluster search. 

There is some additional processing if an insert request generates a new 
type-C subdescriptor. In this situation, the new descriptor-id must be 
received before this predicate is ready for the cluster search (13,14) . After 
the descriptor- id is received, the predicate is ready for the cluster 
search(15). The actual insertion of the new descriptor-id is delayed until 
all descriptors have been determined. The steps required for the insertion of 
the new descriptor-id is described in Appendix D. If there is no new type-C 
subdescriptor, then no wait is necessary (12) . 

The above process is performed for each predicate of the query component 
of the request. At the end of the descriptor search phase, we have found a 
list of descriptor ids for each predicate of the query component. The Carte- 
sian product of the lists of descriptor ids is formed, yielding a list of 
descriptor-id groups for the request. The descriptor-id groups are then 
passed to the third phase of directory management, the cluster search. 
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4.3. The Cluster Search 



The cluster ids of the clusters corresponding to the descriptor-id groups 
of the request must be determined by examining the cluster definition 
table(CDT). The CDT is stored in two parts, a Descriptor-Td-Cluster-Id-Bit- 
Map Table (DCBMT) and a Cluster-Id-to-Secondary-Storage-Address Table (CSSAT) . 
The DCBMT is used during the cluster search phase, while the CSSAT is used 
during the address generation phase. A sample DCBMT is shown in Figure 12. 
Recall that a cluster is defined by a descriptor-id set. There is a bit map 
for each descriptor-id. Each bit map has one bit for each cluster-id in the 
database. A 1 bit corresponding to a cluster means that the descriptor-id 
appears in the definition of that cluster. Thus in the example, the given 
descriptor-id in the bit-map set defining clusters 2,6,11 and 17. 

The bit-map index is used to find the bit-map set for a particular 
descriptor id. The bit-map index is stored in main memory. A bit-map set 
contains pointers to the first set of bits in the bit map for a group of 
descriptor ids. This bit map may be subdivided into several blocks. Thus, 
for each descriptor id, we retrieve one bit-map set and one or more bit-map 
blocks. 

The cluster-ids corresponding to a descriptor-id group are determined by 
logically ANDing together the bit maps for each descriptor- id in the group. 
Thus input for the cluster search is the descriptor-id group. The states and 
transitions of the cluster search phase are shown in Figure 13. 

The cluster search occurs in two steps. First the bit-map sets are 
determined for each descriptor id. Then the bit maps for each descriptor id 
are determined. At this point the bit maps for the descriptor-id groups are 
logically ANDed together to determine the required clusters. The bit-map sets 
are read first so as to avoid reading a bit-map set more than once. 

With slightly more detail in Figure 13, the cluster search proceeds as 
follows. First the bit-map set is read for each descriptor-id (1) . When all 
the bit-map sets have been read (2), the bit maps for each descriptor id can be 
found. The descriptor ids are again processed one at a time. The first block 
of bits from the bit map is read (4). Then any additional bits from the bit 
map are read (5). When all bits have been read, for every descriptor id, the 
cluster search is finished (6). If there is no bit map for this descriptor. 
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then nothing is done (3). 

We process each descriptor-id group of the query in the above manner to 
determine all of the clusters for the request. After the clusters have been 
determined by the cluster search and have been locked by concurrency control, 
it is time to determine the disk address (es) of the data records. 

. The Address Generation 

This phase, the last phase of directory management, is called the address 
generation. As mentioned in the last section the disk addresses are stored in 
the Cluster-Id-to-Secondary-Storage-Address Table (CSSAT) as shown in the exam- 
ple in Figure 14. Each cluster has a fixed number of addresses stored in a 
cluster-address set, two in the example. Additional addresses are stored in 
an overflow area. 

There are two cases to consider, the processing of insert requests and of 
non-insert, i.e., update, delete and retrieve, requests. For non-insert 
requests the disk addresses must be determined so that the appropriate data 
can be read by record processing. The CSSAT does not have to be modified. On 
the other hand, for insert requests it will be necessary to modify the CSSAT, 
either to increment the number of records in a track or to add a new track. 
Thus this case is more complex. 

4.4.1. Hie Address Generation for a Non-insert Request 

Let us first consider the case of a non-insert request. As with the 
cluster search, the address generation for a non-insert request is broken down 
into two steps for efficiency. 

The states and transitions for each cluster are shown in Figure 15. In 
the first step, all the cluster-address sets are determined for each clus- 
ter(l,2). Then, the actual addresses are determined for each cluster. The 
determination of the addresses requires reading the first block of 
addresses(4) and possibly several overflow blocks(5). Processing for this 
cluster is finished when there are no additional addresses to be found (6). If 
there are no records for the cluster in this backend, then, of course, no 
reading is required (3). 
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Figure 14. A Sample Cluster-Id-to-Secondary-Storage-Address Table (CSSAT) 
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4.4.2. The Address Generation for an Insert Request 



When processing an insert request, there is only one cluster to be deter- 
mined. In fact, only one address at which to store the record needs to be 
determined. However, the CSSAT must be updated to reflect the insertion of 
the new record. Reading of the CSSAT is similar to the non-insert case. How- 
ever additional processing is needed to update the CSSAT. The states and 
transitions for each cluster are shown in Figure 16. 

Let us first consider the case where a new cluster is being created. We 
get a track for the new cluster. If no cluster-address set exists, then one 
is created, the new track address is inserted and the cluster-address set is 
written to secondary storage(l). On the other hand, if the cluster-address 
set already exists, it must be read (2). The new track address is inserted and 
the updated cluster-address set written(3). In either case, the address gen- 
eration phase is done (4) . 

Next let us consider the case of an old cluster. In this case an existing 
cluster-address set must be read(2). At this point, processing differs 
depending on whether or not overflow address blocks must be processed. No 
overflow processing is required if the new record fits in the last track 
assigned to the cluster. In this case, the remaining space in the track is 
updated and the cluster-address set is written (3) . A second case also 
requires no processing of overflow blocks. If a new data track is required 
and there is room in the cluster-address set for the new track address, then 
this address is added, the space remaining in the track updated and the 
cluster-address set is written(3). In either case, the address generation 
phase is done after the cluster-address set has been written(4). 

The most complex processing is required when it is necessary to use over- 
flow blocks. Such processing occurs in three cases. The simplest of these 
cases occurs when it is necessary to create the first overflow block. In this 
case the address of the overflow block, the address of the new track, and the 
space remaining in the new track are added to the cluster-address set, which 
is then written(5). The new address is also put in the overflow block and 
that block is written(6). The address generation phase is done(13) when the 
write is finished. 
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Figure 16. Address Generation (insert request) 



The final two cases occur when the last track is full and the cluster in 
question already has one or more overflow address blocks. Since the last 
track is full, a new track is required and the address of that track must be 
added to the end of the overflow addresses of the CSSAT. Processing begins by 
reading the first overflow address(8). If other overflow addresses are 
present, they must also be read (12). If there is room in the last overflow 
block., the new address is added to the overflow block and that block is writ- 
ten to the secondary storage (9). On the other hand, there may be no room in 
the last overflow block. In this case, the address for the new block is 
obtained and a pointer to it stored in the previous last block (10) . After 
that write is finished, the new overflow addresses may be written (11). In 
either case, the address generation phase is done after the last overflow 
block has been written (13) . 

In all of the discussion above, it is important to remember that address 
generation for an insert request occurs at only one backend, the one the con- 
troller has chosen to actually insert the new record. The other backends are 
finished processing the insert after they have determined the cluster for the 
insert and the controller has broadcast the number of the backend which is to 
store the new record. 
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5. AN UPDATED DESCRIPTION OF MDBS MESSAGES 



In this chapter we examine the revisions made to the MDBS message passing 
facilities first described in [Boyn83] . In the MDBS message passing facili- 
ties there are 31 message types and one general message format (shown in Fig- 
ure 17) . This same format is used for each of the three message passing 
facilities, namely, messages within the controller, messages with the backend, 
and messages between computers. Messages between computers are divided into 
two classes, messages between backends, and messages between the controller 
and the backends. Figure 18 describes each of the MDBS message types. 

Comnunication between computers in MDBS is achieved by using a time- 
division-multiplexed bus called the parallel communication link (PCL) 
[DEC79a] . We built a software interface to this bus for each computer con- 
sisting of two complimentary processes. The first process, get_pcl, gets mes- 
sages from other computers off the PCL. The second process, put_pcl, puts 
messages on the bus to be sent to other computers. The controller and each 
backend have their own get_pcl and put_pcl processes. 

In the rest of this chapter, we first present the revised list of MDBS 
message definitions. Then, we examine the sequence of actions for an insert, 
delete, retrieve and update request in the MDBS message passing environment. 



Message Type 
Message Sender 
Message Receiver 
Message Text 



(a numeric code) . 

(a numeric code) . 

(a numeric code) . 

(an alphanumeric field terminated 
by an end of message marker) . 



Figure 17. MDBS General Message Format 



MESSAGE-TYPE NUMBER AND NAME 



1 TRAFFIC UNIT 
REQUEST RESULTS 

NUMBER OF REQUESTS IN A TRANSACTION 
AGGREGATE OPERATORS 
5 REQUESTS WITH ERRORS 
PARSED TRAFFIC UNIT 
NEW DESCRIPTOR ID 
BACKEND NUMBER 
CLUSTER ID 

10 REQUEST FOR NEW DESCRIPTOR ID 
BACKEND RESULTS FOR A REQUEST 
BACKEND AGGREGATE OPERATOR RESULTS 
RECORD THAT HAS CHANGED CLUSTER 
RESULTS OF A RETRIEVE OR FETCH 
CAUSED BY AN UPDATE 
15 DESCRIPTOR IDS 

REQUEST AND DISK ADDRESSES 
CHANGED CLUSTER RESPONSE 
FETCH 

OLD AND NEW VALUES OF ATTRIBUTE 
BEING MODIFIED 

20 TYPE-C ATTRIBUTES FOR A TRAFFIC UNIT 
DESC-ID GROUPS FOR A TRAFFIC UNIT 
CLUSTER IDS FOR A TRAFFIC UNIT 
RELEASE ATTRIBUTE 

RELEASE ALL ATTRIBUTES FOR AN INSERT 
25 RELEASE DESCRIPTOR-ID GROUPS 
ATTRIBUTE LOCKED 
DESCRIPTOR-ID GROUPS LOCKED 
CLUSTER IDS LOCKED 
29 NO MORE GENERATED INSERTS 
29 NO MORE GENERATED INSERTS 

29 NO MORE GENERATED INSERTS 

30 REQUEST ID OF A FINISHED REQUEST 

31 AN UPDATE REQUEST HAS FINISHED 
31 AN UPDATE REQUEST HAS FINISHED 
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J5. .1 . Revised Definitions of MDBS Messages 

In this section we give short descriptions of the revised definitions of 
MDBS messages. The first group of messages are those between the host and the 
controller and within the controller itself. These messages are shown in Fig- 
ure 19. 



Message type : (1) Host Traffic Unit 
Source : Host 

Destination : Request Preparation 

Explanation : The traffic unit represents a single request or 
transaction from a user at the host machine. 



Message type : (2) Request Results 
Source : Post Processing 
Destination : Host 

Explanation : Contains the results for a request after being 
collected from all the backends and aggregated 
if necessary. 



Message type : (3) Number of Requests in a Transaction 
Source : Request Preparation 
Destination : Post Processing 

Explanation : Request Preparation sends to Post Processing 
the number of requests in a traffic unit. 

This enables Post Processing to determine whether the 
processing of a traffic unit is complete. 



Message type : (4) Aggregate Operators 
Source : Request Preparation 
Destination : Post Processing 

Explanation : Request Preparation sends the aggregate 
operators to Post Processing. 



Message type : (5) Requests with Errors 
Source : Request Preparation 
Destination : Post Processing 

Explanation : Requests with errors will be found in 
Request Preparation by the Parser 
and sent to the Post Processing 
directly. Post Processing will send 
the requests with errors Back to the host. 



The next set of messages deals with the communication between the con- 
troller and the Directory Management process within each backend. These mes- 
sages can be found in Figure 20. 



Message type : (6) Parsed Traffic Unit 
Source : Request Preparation 
Destination : Directory Management 

Explanation : This is the prepared traffic unit sent by 
Request Preparation. 
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Figure 19. Controller Related Messages 
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Figure 20. REQP, IIG (Controller), and DM (Backend) Related Messages 
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Message type : (29) No More Generated Inserts 
Source : Request Preparation 
Destination : Directory Management 

Explanation : This message indicates that insert request for all 
the records that have changed cluster as a result 
of an update request have been generated and sent 
to Directory Management. 



Message type : (7) New Descriptor Id 

Source : Insert Information Generation 
Destination : Directory Management 

Explanation : This message is a response to the Directory Management 
request for a new descriptor id. 



Message type : (8) Backend Number 

Source : Insert Information Generation 
Destination : Directory Management 

Explanation : This message is used to specify which backend is to 
insert a record. 



Message type : (9) Cluster Id 

Source : Directory Management 
Destination : Insert Information Generation 

Explanation : Directory Management sends a cluster id to Insert 
Information Generation for an insert request. IIG 
will decide where to do the insert. 



Message type : (10) Request for New Descriptor Id 
Source : Directory Management 
Destination : Insert Information Generation 

Explanation : When Directory Management has found a new descriptor it 
is sent to Insert Information Generation 
to generate an id. 



The third group of messages deal with the flow from the Record Processing 
process in a backend to the Post Processing and Request Preparation processes 
in the controller. Figure 21 shows the flow of these messages. 



Message type : (11) Results of a Request from a Backend 
Source : Record Processing 
Destination : Post Processing 

Explanation : This message contains the results that a specific backend 
found for a request. 



Message type : (12) Aggregate Operator Results from a Backend 
Source : Record Processing 
Destination : Post Processing 

Explanation : When an aggregate operation needs to be done on the 

retrieved records, each backend will do as much aggregation 
as possible in the aggregate operation function of Record 
Processing. This message carries those results to 
Post Processing. 
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Figure 21. REQP, RECP and PP Related Messages 
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Message type : (13) Record that has Changed Cluster 
Source : Record Processing 
Destination : Request Preparation 

Explanation : This message is a record which has changed cluster, 

Request Preparation will prepare it as an insertion and 
send it to the backends. 



Message type : (29) No More Generated Inserts 
Source : Record Processing 
Destination : Reguest Preparation 

Explanation : This message indicates that all the records that have 
changed cluster as a result of an update request have 
been sent to Request Preparation. 



Message type : (14) Results of a Retrieve or Fetch Caused by an Update 
Source : Record Processing 
Destination : Reguest Preparation 

Explanation : This message carries the information from a retrieve or 
fetch back to Request Preparation to complete an 
update with type-III or type-IV modifier. 



The following descriptions are for messages between Directory Management 
processes residing on different backends and between Directory Management and 
Record Processing within a backend. These messages are shown in Figure 22. 



Message type : (15) Descriptor Ids 
Source : Directory Management 
Destination : Directory Management (other backends) 
Explanation : This message contains the results of descriptor 
search by Directory Management. 



Message type : (16) Request and Disk Addresses 
Source : Directory Management 
Destination : Record Processing 

Explanation : This message contains a request and disk addresses 

for Record Processing to come up with the results for 
the request. 



Message type : (17) Changed Cluster Response 
Source : Directory Management 
Destination : Record Processing 

Explanation : Directory Management uses this message to tell 

Record Processing whether an updated record has changed 
cluster. 



Message type : (29) No More Generated Inserts 
Source : Directory Management 
Destination : Record Processing 

Explanation : This message indicates that all insert requests 
generated as a result of an update request have 
been sent to Record Processing. 
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Figure 22. DM and RECP Related Messages 



Message type : (18) Fetch 

Source : Directory Management 
Destination : Record Processing 

Explanation : Fetch is a special retrieval of information for Request 
Preparation due to an update request with type-IV 
modifier. 



Message Type : (19) Old and New Values of Attribute being Modified 
Source : Record Processing 
Destination : Directory Management 

Explanation : Record Processing uses this message to check whether a 
record that has been updated has changed cluster. 



Message Type : (31) An Update Request has Finished 
Source : Record Processing 
Destination : Directory Management 

Explanation : Record Processing signals Directory Management 
that an update request has finished execution. 



The last set of messages are the Concurrency Control related messages. 
These messages pass information from either Directory Management or Record 
Processing to Concurrency Control. These are shown in Figure 23. 



Message Type 
Source 
Destination 
Explanation 



(20) Type-C Attributes for a Traffic Unit 
Directory Management 
Concurrency Control 

Concurrency Control takes the attributes in this 
message and determines when Descriptor Search for 
an attribute can be performed. 



Message Type 
Source 
Destination 
Explanation 



(21) Descriptor-id Groups for a Traffic Unit 
Directory Management 
Concurrency Control 

Concurrency Control takes the descriptor-id groups 
in this message and determines when Cluster Search 
for a request can be performed. 



Message Type 
Source 
Destination 
Explanation 



(22) Cluster Ids for a Traffic Unit 
Directory Management 
Concurrency Control 

Concurrency Control takes the cluster ids in this 
message and determines when a request can continue with 
Address Generation and the rest of request execution. 



Message Type 
Source 
Destination 
Explanation 



(23) Release Attribute 
Directory Management 
Concurrency Control 

Directory Management uses this message to signal 
Concurrency Control that a request has performed 
Descriptor Search on an attribute, and the lock on 
the attribute held by the request can be released. 
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Figure 23. DM, RECP, and CC Related Messages 
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Message Type : 
Source : 
Destination : 
Explanation : 



(24) Release All the Attributes for an Insert 
Directory Management 
Concurrency Control 

Directory Management uses this message to signal 
Concurrency Control that an insert request has 
performed Descriptor Search on all the attributes, and 
the locks on the attributes held by the request can 
be released. 



Message Type : (25) Release Descriptor-Id Groups 

Source _ : Directory Management 

Destination : Concurrency Control 

Explanation : Directory Management uses this message to signal 

Concurrency Control that an insert request has 
performed Cluster Search for a request, and the locks 
on the descriptor-id groups held by the request can 
be released. 



Message Type 
Source 
Destination 
Explanation 



(31) An Update Request Has Finished 
Directory Management 
Concurrency Control 

Directory Management uses this message to signal 
Concurrency Control that an update request has 
finished execution, and all the locks neld by the 
request can be released. 



Message Type : 
Source : 
Destination : 
Explanation : 



(26) Attribute Locked 
Directory Management 
Concurrency Control 

Concurrency Control signals Directory Management 
that an attribute is locked for a request, and 
Descriptor Search can be performed. 



Message Type 
Source 
Destination 
Explanation 



(27) Descriptor-Id Groups Locked 
Directory Management 
Concurrency Control 

Concurrency Control signals Directory Management 
that the Descriptor-id groups needed by a request 
are locked, and Cluster Search can be performed. 



Message Type : 
Source : 
Destination : 
Explanation : 



(28) Cluster Ids Locked 
Directory Management 
Concurrency Control 

Concurrency Control signals Directory Management that 
the cluster ids needed by a request can continue with 
address Generation and tne rest of request execution. 



Message Type 
Source 
Destination 
Explanation 



(23) Request Id of a Finished Request 
Record Processing 
Concurrency Control 

Record Processing signal? Concurrency Control that a 
non-update request has finished execution, and the 
locks on cluster ids held by the request can be 
released. 
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_5 ._2. Request Execution in MDBS - Viewed Vie Message Passing 

In this section, we describe the sequence of actions for a request as it 
moves through MDBS. The sequence of actions will be described in terms of the 
types of messages passed between the MDBS processes: Request Preparation 
(REQP) , Insert Information Generation (IIG) , Post Processing (PP) , Directory 
Management (DM) , Record Processing (RECP) and Concurrency Control (CC) . The 
order in which messages are passed will be denoted alphabetically ('a' is 
first) . The digit following the ordering letter will be the message number as 
shown in Figure 18. We examine the four types of requests, insert, delete, 
retrieve, and update. 

5.2.1. Sequence of Actions for an Insert Request 

The sequence of actions for an insert request is shown in Figure 24. The 
traffic unit (al) comes into REQP from the host carrying an insert request. 
REQP sends to PP the number of requests in the traffic unit (b3) . After 
preparation, the formatted request is sent to DM from REQP (c6) . DM sends the 
type-C attributes needed by the request to CC (d20) . Once an attribute is 
locked, and Descriptor Search can be performed, CC signals DM (e26) . DM will 
then perform Descriptor Search. From DM, descriptor ids for the request will 
be sent to the other backends in the MDBS system (f 15) . DM also signals CC to 
release the locks on attributes (g24) . The descriptor ids found by the other 
backends will be received by DM (hl5) . DM now sends the descriptor-id group 
for the request to CC (121) . Once the descriptor-id group is locked and Clus- 
ter Search can be performed, CC signals DM (j27) . DM will then perform Clus- 
ter Search. To determine where the insert will occur, DM will send the insert 
cluster id to IIG (k9) . Once the backend has been selected, IIG will send the 
backend number to DM (m8) . DM updates its directory tables if needed, and 
signals CC to release the lock held by the request on the descriptor-id group 
(n25) . DM will send the insert cluster id to CC (o22) . CC will respond to DM 
when the insert request can proceed with Address Generation and the rest of 
request execution (p28) . With the go ahead from CC, DM will perform Address 
Generation and send RECP the request and its required disk address (ql6) . 
After the insert has occurred, RECP will notify CC that the request is done 
(r30) , followed by a message to PP that the request has completed (sll) . PP 
will finish the processing by sending a results message to the host (t2) . 
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Figure 24. Sequence of Messages for an Insert Request 
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5.2.2. Sequence of Actions for a Delete Request 

The sequence of actions for a delete request is shown in Figure 25. A 
traffic unit is sent to REQP from the host containing the delete request (al) . 
REQP notifies PP of the number of requests in the traffic unit (b3) . Next, 
REQP sends the request down to DM (c6) . DM sends the type-C attributes needed 
by the request to CC(d20) . Once an attribute is locked and Descriptor Search 
can be performed, CC signals DM (e26) . DM performs Descriptor Search and sig- 
nals CC to release the lock on the attribute (f23) . The descriptor ids for the 
request are next sent to the other backends from DM (gl5) . The other backends 
respond with the descriptor ids they have found (hi 5) . DM sends the 
descriptor-id groups for the request to CC (i21) . Once the descriptor-id 
groups are locked and Cluster Search can be performed, CC signals DM ( j27) . DM 
performs Cluster Search and signals CC to release the locks on the 
descriptor-id groups (k25) . DM will next send the cluster ids for the delete 
request to CC (m22) . Once the cluster ids are locked and the request can con- 
tinue with Address Generation and the rest of request execution, CC signals DM 
(n28) . DM will then perform Address Generation and send to RECP the address 
and the request (pl6) . After RECP has performed the delete request, it will 
notify CC that the request is through (p30) . PP will then receive a results 
message from RECP telling it that the request is done (qll) . PP will then 
notify the host with a results message (r2) . 

5.2.3. Sequence of Actions for a Retrieve Request with Aggregate Operator 

The sequence of actions for a retrieve request is shown in Figure 26. 
First the retrieve request comes to REQP from the host (al) . REQP sends two 
messages to PP: the number of requests in the transaction (b3) and the aggre- 
gate operator of the request (c4) . The third message sent by REQP is the 
parsed traffic unit which goes to DM in the backends (d6) . DM sends the 
type-C attributes needed by the request to CC (e20) . Once an attribute is 
locked and CC can be performed, CC signals DM ( f 26 ) . DM will then perform 
Descriptor Search and signal CC to release the lock on that attribute (g23) . 
DM will send the descriptor ids for the request to the other backends (hl5) . 
The DM processes in the other backends will send their descriptor ids to the 
DM process residing in this backend ( i 15) . DM now sends the descriptor-id 
groups for the retrieve request to CC ( j 21 ) . Once the descriptor-id groups 
are locked and Cluster Search can be performed, CC signals DM (k27). DM will 
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Figure 25. Sequences of Messages for a Delete Request 
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Figure 26. Sequence of Messages for a Retrieve Request 
with Aggregate Operations 
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then perform Cluster Search and signal CC to release the locks on the 
descriptor-id groups (m25) . Next, DM will send the cluster ids for the 
retrieval to CC (n22) . Once the cluster ids are locked, and the request can 
proceed with Address Generation and the rest of the request execution, CC sig- 
nals DM (o28) . DM will then perform Address Generation and send the retrieve 
request and the addresses to RECP (pl6) . Once the retrieval request has exe- 
cuted properly, RECP will tell CC that the request is done and the locks on 
the cluster ids can be release (q30) . After the retrieval results have been 
aggregated in the backend, that result will be sent to PP for further aggrega- 
tion (rl2) . When PP is done, the final results will be sent to the host (s2) . 

5.2.4. Sequence of Actions for an Update Request Causing a Change in Clus- 
ter 



The sequence of actions for an update request that causes a record to 
change cluster is shown in Figure 27. This request is processed in two parts. 
First, after processing the update, it is determined that a record has changed 
cluster. Then, an insert is generated to actually store the new record. As in 
the previous examples, we will go through the complete execution of this 
request. 

The host sends the update request to REQP (al) . REQP follows through by 
formatting the request and sending PP the number of requests in the transac- 
tion (b3) . DM also receives a message from REQP, the parsed traffic unit (c6) . 
DM sends the type-C attributes needed by the request to CC (d20) . Once an 
attribute is locked and Descriptor Search can be performed, OC signals DM 
(e29) . DM will then perform Descriptor Search and send a message to CC to 
release the lock on the attribute (f23) . The DM in each backend will exchange 
descriptor ids with each of the other backends (gl5 and hl5) . DM sends the 
descriptor-id groups needed by the update request to CC (i21) . Once the 
descriptor-id groups are locked and Cluster Search can be performed, CC sig- 
nals DM ( j 27 ) . DM will then perform Cluster Search. DM will send the cluster 
ids to CC to check if the request can continue with Address Generation and the 
rest of request execution (k22) . Once CC responds to DM that the request can 
go through (m28) , DM will generate the disk addresses and send the request as 
well as the addresses to RECP(nl6) . When RECP retrieves the old values of the 
attribute being modified by the update, it will then send these old values and 
the new values to DM to check for records that have changed cluster (ol9) . A 
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Figure 27. Sequence of Messages for an Update Request 
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reply will be sent to RECP from DM stating (for our example) that the update 
does cause a record to change cluster (pl7) . The change of cluster by a record 
requires an insert, therefore RECP will send the record that has changed clus- 
ter to REQP (ql3) . REQP will then generate an insert request. After sending 
all the updated records that have changed cluster to REQP, REQP sends a mes- 
sage to REQP (r29) indicating that there are no more changed-cluster records 
at this backend. (This message and the next message are needed to insure that 
the updated records are not inserted in the backends before the update 
requested has finished updating all the records. If the changed-cluster 
records are inserted too early, the update request may update some of them 
again.) After receiving the message (r29) from RECP in all the backends and 
after generating all the required insert requests, REQP sends a message to DM 
indicating that there will not be any more insert requests for this update 
request (s29) . After receiving the message (s29) and performing directory- 
management processing for all the generated insert requests, DM sends a mes- 
sage to RECP indicating that there will not be any more insert request for 
this update request (t29) . (RECP needs this message to determine when the 
update request is completely done. The update request is completely done when 
all the insert requests caused by it are done.) 

Let us now describe how the generated insert is processed. The execution 
of this request proceeds as other insert requests. REQP sends DM the parsed 
traffic unit for the insert (u6) . DM sends the type-C attributes needed by the 
insert request to CC (v20) . Once an attribute is locked and Descriptor Search 
can be performed, CC signals DM (w26) . DM will then perform Descriptor 
Search. From the DM, descriptor ids for the request will be sent to the other 
backends in the MDBS system (xl5) . DM also signals CC to release the locks on 
attributes (y24) . The descriptor ids found by the other backends will be 
received by DM (zl5) . DM now sends the descriptor-id group for the request to 
CC (A21) . (Note that we are now using capital letters for sequencing.) Once 
the descriptor-id group is locked and Cluster Search can be performed, CC sig- 
nals DM (B27) . DM will then perform Cluster Search. To determine where the 
insert will occur, DM will send the insert cluster id to IIG (C9) . Once the 
backend has been selected, IIG will send the backend number to DM (D8) . DM 
updates its directory tables if needed, and signals CC to release the lock 
held by the insert request on the descriptor-id group (E25) . DM will send the 
insert cluster id to CC (F22) . CC will respond to DM when the insert request 
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can proceed with Address Generation and the rest of the request execution 
(G28) . With the go ahead from CC, DM will perform Address Generation and send 
RECP the request and its required disk address (H16) . After the insert has 
occurred. RECP will notify CC that the insert request is done (130) . 

After executing all the insert requests caused by the update request, 
RECP signals DM that the update request is completely done (J31) . DM will free 
the space used by the update request and signal CC that the update request has 
finished (K31) . RECP also sends the results of the update request to PP (Lll) 
and PP notifies the host that the update has completed (M2) . 
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6. CONCLUSIONS AND FUTURE PLANS 



This report concludes the series of reports [Kerr82, He82, Boyn83] on 
the implementation of MDBS. We have finished the design, coding, implementa- 
tion and testing of Version F which is the target version envisioned from the 
outset. Thus we can begin the next phases of experimentation, specifically, 
design verification and performance evaluation. To support these new phases 
of experimentation, we need to increase the number of backends in the system 
to, at least, six. The issues involving the hardware reconfiguration and 
expansion of MDBS are discussed in Section 6.1. 

Additionally, we are investigating a security mechanism and language 
interfaces for relational and hierarchical data manipulation languages. These 
topics, along with the design verification and performance evaluation issues, 
are examined in Section 6.2. Finally, Section 6.3 contains a long-range goal 
of the Laboratory for Database Systems Research, i.e., the development of a 
general methodology for benchmarking database machines and database software 
systems. 

6^_1 . Hardware Reconfiguration for MDBS 

The current hardware configuration of MDBS consists of a VAX-11/780 run- 
ning as the controller and two PDP-ll/44s running as backends. Intercomputer 
communication is supported by three parallel communication links (PCL-llBs) . 
To increase the number of backends to six, we would need to purchase four more 
PDP-ll/44s and four more PCL-llBs. However, this has not been our original 
plan. Our plan was to use the Ethernet-like communications link and the smal- 
lest and cheapest minis which can support hard disks for our configuration. As 
always in the computer field, the intended hardware was not available in 1980. 
Since then we have used more powerful hardware, the PDP-ll/44s and VAX-11/780 
and more awkward hardware such as the PCL-llBs. Thus, we are now investigating 
the possibility of replacing our current configuration with newer and more 
appropriate hardware. In particular, we are thinking of replacing our back- 
ends with the Digital LSI-11 series, either the 11/23, the 11/23+, or the 
11/73. The gain is in cost and service. The cost of the LSI-11/23 or LSI- 
11/23+ is about half the cost of a PDP-11/44. The cost of the LSI-11/73 is 
about two-thirds the cost of a PDP-11/44. Since the software is portable, 
there is no problem in down-loading the existing software onto any of these 
three machines. 
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We are also considering a change in the communications hardware. When 
the implementation of MDBS began, the technology for local-area networks, 
e.g., Ethernet [Metc76] , was not available. The replacement of the PCL 
hardware with an Ethernet or Ethernet-compatible network would standardize the 
communications hardware. Additionally, unlike Ethernet, the PCL is not a 
broadcast bus. We have required a broadcast bus for MDBS. In the current 
environment, when the controller needs to broadcast a message to all the back- 
ends, it must send the message separately to each backend. In other words, if 
there are two backends, the controller sends two messages. Thus, the 
message-passing overhead increases as the number of backends increases. An 
Ethernet will eliminate this overhead. 

§_. 2 _. New Research 

The new research on MDBS involves three major areas, a security mechan- 
ism, language interfaces to support the relational and hierarchical data mani- 
pulation languages, and the performance evaluation of the MDBS. 

6.2.1. A Security Mechanism 

Since security is an integral part of a database system, the design of a 
security mechanism is mandated. The design considerations of a security 
mechanism consists of two parts. First, the level (s), known as granule(s), at 
which the security control is applied must be determined. In MDBS there are 
four possibilities: the attribute level, the descriptor level, the cluster 
level, and the record level. Second, given the level (s) at which security is 
defined, the security mechanism is then specified. This is not an easy task, 
since the directory information about the levels changes dynamically when a 
new type-C descriptor or a new cluster is created. 

6.2.2. Language Interfaces 

There are three separate projects underway involving language interfaces. 
In designing language interfaces, we are providing the user access to MDBS 
using a variety of data manipulation languages. The series of papers [Bane77, 
Bane78, Bane80] demonstrated that a relational, hierarchical or network data- 
base can be transformed into an attribute-based database. Thus it is reason- 
able to design a language interface which maps a given data manipulation 
language into the attribute-based query language, so that the user may use the 
given language on the transformed database. One project involves the design 
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of a language interface for the relational query language, SQL [Astr76] . A 
second project involves the design of a language interface for DL/I of IMS 
[McGe77] . The third project is considering the various algorithms which can 
be used to implement the sort and join operations in the attribute-based sys- 
tem for the relational language interface. 

6.2.3. Performance Evaluation 

There are two projects dealing with the performance evaluation of MDBS. 
The internal performance evaluation project is measuring the execution times 
of the modules of the backend. These measurements include the time to process 
a particular message, the disk I/O time, the intra-computer and inter-computer 
message passing times, the process switch time, etc. The external performance 
evaluation project is measuring the throughput of the system. The throughput 
of MDBS is defined as the average number of user requests executed by the sys- 
tem in a second [Hsia81a] . The throughput of the system can be obtained for 
the four primary operations in MDBS. 

What 's Next 

The work on performance evaluation and the relational and hierarchical 
language interfaces leads toward the ultimate goal of our research efforts: 
the specification of a general methodology to benchmark database machines and 
database software systems. We intend to extend the earlier work on benchmark- 
ing database machines and software systems which support the relational model 
[Stra84] , to benchmark systems and machines based on the hierarchical and net- 
work data models. 

Additionally, we intend to permit the comparison of two or more database 
systems. The benefit to such an approach is the ability to easily benchmark 
and compare similar and dissimilar database systems, i.e., a relative com- 
parison . However, this approach does not preclude absolute comparis on where 
only a single system has to be benchmarked. In this case, the benchmarks are 
of course written in the given data manipulation language. 
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APPENDIX A 

HOW TO READ AND FOLLOW THE PROGRAM SPECIFICATIONS 

The appendices in this series have contained the detailed design of MDBS. 
In Appendix B, the programs for the directory management concurrency control 
are described and specified. In Appendix C, the programs for directory 
management using secondary memory are described and specified. These programs 
represent those parts of MDBS that have been newly designed and redesigned, 
since the first three reports in this series were written. 



A.l_ Parts within an Appendix 

Each appendix begins with a introduction which outlines the major com- 
ponents of the design. For example, the design of the controller subsystem, 
presented in [He82] , consisted of three major parts: request preparation, 
insert information generation and post processing. The design of a backend 
subsystem also consists of three major parts: directory management, record 
processing and concurrency control. Primary-memory-based directory management 
and record processing, were presented in the previous reports. The third 
part, namely, concurrency control, in presented in Appendix B of this report. 
Finally, the revisions for secondary-memory-based directory management are 
presented in Appendix C. 



A._2 The Format of a Part 

In each part, we provide the following documentation elements: 

(1) Title of the part, 

(2) Name of the design, 

(3) Name of the designer, 

(4) Date the design was first submitted, 

(5) Dates of design modifications, 

(6) Statements of the design purpose, and of the input and output 
requirements, 

(7) Formal specifications of the input and output, if necessary, 

(8) Procedure names used in the design, 

(9) Jackson chart of the design, if necessary, 

(10) Data structures used in the design, 
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(11) Program specification of the design. 



h.3_ Documentation Techniques for ci Part 

In the previous section, we listed the various documentation elements. 
They are used to describe a design. Documentation elements 1 through 5 are 
written in English phrases. Document element 6 is written in prose. On the 
other hand, document elements 7 through 11 can be expressed more effectively 
using other means. Specifically, we have used Backus-Naur form (BNF) for 
writing the specifications in document element 7. 

The procedure names of document element 8 are shown in a program hierar- 
chy. The use of the hierarchy makes clear the calling sequences of the pro- 
cedures named. The data structures of documentation element 10 are specified 
in either the system specification language (SSL) or in the C programming 
language. In documentation element 11, the procedures, themselves, are speci- 
fied in SSL. 

Except for the programming team that writes the procedures, other teams 
will usually not be interested in the internal logic of the procedures. Con- 
sequently, they need only know the higher-level specifications of the pro- 
cedures. The SSL employed in MDBS is an ideal specification language for 
revealing the design of the procedures from a top-to-bottom-and-layer-to-layer 
way. It also works well with the hierarchical organization of procedures. 
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APPENDIX B 



THE SSL SPECIFICATIONS FOR DIRECTORY MANAGEMENT CONCURRENCY CONTROL 



The system specification for directory management concurrency control is 
given in this appendix. 



/* (1) Directory Management Concurrency Control 

/* 2 Design: s DM CC 

/* (3) Designers : A.^Orooji, D. S. Kerr 

/* 4 Date : July 24, 1983 

/* (5) Modified : December 28,1983 

/* (6) Purpose : 

/* The directory data, namely the descriptor-to-descriptor-id 

/* table (DDIT) and the cluster-definition table (CDT) , may be 
/* modified during request execution. Thus before descriptor search (DS) 
/* can access DDIT or cluster search (CS) can access CDT, appropriate 
/* locks must be obtained. 

/* There are two types of locks: READ and WRITE. A type-C attribute 

/* must be locked before a request can perform DS on that attribute. 

/* To avoid deadlock the type-C attributes are sorted before being 
/* sent to DM CC. 

/* All de3criptor-id-groups needed by a request must be locked before 

/* the request can perform CS. Each descriptor-id group needed by a 
/* request is sorted (before being sent to DM CC) , and all the 
/* descriptor-id groups needed by a request are sorted. 



*/ 

*/ 

V 

V 

V 

V 
*/ 

V 

V 

V 

V 

V 

V 

V 

V 
*/ 

V 
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(8) Procedure Hierarchy for DM_CC 



DM CC 

T 

r r 

DM CC_ DM CC R$ DM CC R$ 

inTt Message Type 



I 

DSCC 
NewTraf 
Unit 
I 

r H i 

DSCC | 
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Unit ~ 
Init - 



DM CC R$ 
Atfrs 



— + 

I 

DSCC 

Complete 



I '+~ 

DM CC_R$ 
ReTease 
Attr 



H 



■+ 



H h 

DSOC 

Insert 

Complefe 

I 

-t— H I- 

I 

DM_CC R$ 

Inserf 

All 



Attr 

Release 




CC 

Update 

Finished 

I 

+ +-+ f- 

I 

DM CC_R$ 

Update 

Finished 

Rid 



+ 
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Release 

Attributes 

I 



H 
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NewTraf 

Unit 

I 

H h + 
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DM CC R$ 
DeSCldG roups 

+-+ 



+ — + 

I 

CSCC 

Complete 



DM CC_R$ 
ReTease 
DidG roups 



+ 1- 



H 

I 

CSCC 

Release 

DidGroups 



-+ 



I 

DSCC_ 

Release 

Attr 

I 

H 1- 



DS Try 
to - ‘ 

start 




Lock - 

Conversion 



CS Try 
to 

Start 

I 



Request 
CSCC_ - 
Lock 

Conversion 



CSCC_ 

Lock 
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(10) Data Structures 



List of abbreviations: 

ATUT - attribute-to-traff ic-unit table 

CS - cluster search 

DM - directory management 

DM_CC - directory management concurrency control 

DS - descriptor search 

TU - traffic unit 

TUAT - traff ic-unit- to-attribute table 

TUDIGT - traff ic-unit-to-descriptor-id-groups table 

Traffic-Unit-to-Attribute Table (TUAT): 

This table contains a list of traffic units and the type-C attributes 
needed by the requests in the traffic units. A type-C attribute needed 
by a request is locked before Descriptor Search (attribute-being-modified 
in an update request is also locked if it is type C) . 

<- TUs that arrived earlier, TUs that arrived later -> 

I TUI T > I TU2~| > ~TU3~T > ... 



V 

T R1 



I 

V 



--> | R2 

__H 

V 



I 

V 



--> | R3 
_____ 

V 



--> ... 



V 



1 Attrl | 


I Attr2 


1 1 Attr3 


1 


| R/W 


! > | R/W 


1 > 1 R/W 


1 > ... 



Attribute-to-Traff ic-Unit Table (ATUT) : 

This table has the same information as TUAT, but it is based on 
attributes. 



I Attrl | 



> | TUI | > | TU2 | > | TU3 I > ... 



1 R1 


1 


R2 


1 R3 


1 


1 R/W 


| > 


R/W 


> [ R/W 


I > ... 



I Attr2 I > ... 
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Traffic-Unit-Descriptor-Id-Groups Table (TUDIGT) : 



This table contains a list of traffic units and the descriptor-id groups 
needed by the requests in the traffic units. The descriptor-id groups 
needed by a request are locked before Cluster Search. 

<- TUs that finished DS earlier, TUs that finished DS later -> 



I TUI | > | TU2 | > | TU3 I > ... 



V 

T Rl | > T R2 | > T R3 T > ... 



• • t # • • 

V 



desc-id 


> 


desc-id 


> 


desc-id 


groupl 




group2 




group3 


TBU/BU 




TBU/BU 




TBU/BU 


R/W 




R/W 




R/W 



V 



(11) Program Specifications 



1 

2 

3 

4 

5 

6 

7 

3 

9 

10 

11 

12 



task () 

{ 



/* DM CC (directory management concurrency control) */ 



/* do initialization */ 

DM CC init() ; 
while - ( TRUE ) 

{ 

/ * get the next message for DM CC 
DM_CC R$Message() ; 

/* get" the message type */ 

MsgType = DM CC R$Type ( ) ; 
switch ( MsgType ) 



V 



case DS NewTU: /* a new traffic unit and the type-C 

attributes needed by the requests in it */ 
DSCC NewTrafUnitO ; 
break"; 



13 case CS NewTU: /* a new traffic unit and the descriptor-id 

— groups needed by the requests in it */ 

14 CSCC NewTrafUnitO ; 

15 brea1<; 

16 case DS ReleaseAttr: /* a type-C attribute needed in DS is 

— released */ 

17 DSCC Complete (); 

18 break"; 

case DS InsertReleaseAllAttrs : /* DS 

the 



19 

20 
21 



DSCC InsertComplete () ; 
break - ; 



for insert releasing all 
type_C attributes */ 



22 

23 

24 

25 

26 

27 

28 

29 

30 

26.1 

26.2 

26.3 

26.4 

26.5 



26.6 

11.1 



11.2 



11.3 

11.4 

11.5 



case CS_ReleaseDidGroups : /* the descriptor-id groups needed 

in CS are released */ 

CSCC Complete ( ) ; 
breath- 

case UpdateFinished : /* An update is finished, so release 

update attribute(if any), 
descriptor-id groups and cluster */ 

CC_UpdateFinished () ; 
break; 



}/* end switch */ 
}/* end while */ 

}/* end main() */ 



CC UpdateFinished () 

/* An update is finished, so release update attribute (if any), */ 

/* descriptor-id groups and clusters */ 

/* receive the request id */ 

DM_CC_R$UpdateFinishedRid ( FinishedRid ); 

/* find and remove update attribute, if any, from TUAT & ATUT */ 
DSCC_InsertUpdateReleaseAttributes( FinishedRid ) ; 

/* find and remove all descriptor-id groups from TUDIGT */ 
CSCC_ReleaseDidGroups ( FinishedRid ); 

/* find and remove all cluster-ids from TUCT and CTUT */ 

/* (included here only for completeness, this is actually */ 
/* part of database concurrency control, NOT directory */ 
/* management concurrency control) */ 

} /* end CC_UpdateFinished */ 



DSCC NewTrafUnitO 

/*" This routine is used when a new traffic unit and the type-C */ 
/* attributes needed by the requests in it are sent from DM to */ 
/* DM CC . */ 
/* This routine adds the traffic unit to TUAT and ATUT, and it */ 
/* tries to start DS. */ 



{ 



/* The message contains a traffic unit and the type-C attributes 
needed by the requests in it. The message is 



{ [ndl, (attrli , attrlj, ...)1, 
[rid2, (attr2i , attr2j, ...)], 



} 



V 



/* Receive the traffic unit and the type-C attributes. */ 

/* Enter the requests and the type-C attributes needed by them 
into TUAT and ATUT. */ 

DS_Traff ic_Unit_Init( FirstRid , FirstAttribute ) ; 

/* Try to convert locks on as many attributes as possible. */ 
DS_Try_to_Start ( FirstRid , FirstAttribute ); 

}/* end DSCC NewTrafUnit */ 
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17.1 

17.2 

17.3 

17.4 



DSCC_Complete () 

/* a type-C attribute needed in DS is released */ 

/* Receive the request id and the type-C attribute. */ 
DM CC R$ReleaseAttr ( ReleasingRid, ReleasedAttr) ; 
/* Remove the attribute from TUAT and ATUT, 

and try to start DS for pther request (s). */ 
DSCC ReleaseAttr( ReleasingRid, ReleasedAttr); 



17.5 } /* end DSCC_Complete */ 



20 .1 DSCC_InsertComplete () 

2 q 2 { /* DS for insert is releasing all the type-C attributes */ 

/* Receive the request id. */ 

20.3 DM CC R$InsertAllAttrRelease ( ReleasingRid ); 

/* Release the attributes. */ 

20.4 DSCC_InsertUpdateReleaseAttrs ( ReleasingRid ); 

20.5 } /* end DSCC_InsertComplete */ 



11.3.1 D6 Traffic Unit Init( FirstRid , FirstAttr ) 

~ /* Setup TUAT and ATUT entries for the new traffic unit. Return */ 
/* pointers to the traffic unit, 1st request, and the position */ 
/* of the first attribute for that request. */ 

11.3.2 { 

/* Receive the traffic unit and the type-C attributes. */ 

11.3.3 DM CC R$Attrs ( ... ); 



11.3.4 

11.3.5 

11.3.6 

11.3.7 

11.3.8 



11.3.9 



/* Enter the requests and the type-C attributes needed by them into 
TUAT and ATUT. */ 

for each request in the traffic unit 

fo^ each attribute needed by this request 
add an entry to TUAT; 

/* the entry contains the following information 
attribute name or id 
mode (R/W) */ 

add an entry to ATUT; 

/* the entry contains the following information 
rid 

mode (R/W) */ 



11.3.10 



}/* end for */ 



11.3.11 



}/* end for */ 



11.3.12 return first request id and first attribute 



11.3.13 } /* end DS Traffic Unit Init */ 
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C=ll. 

C.l 



C.2 

C.3 

C.4 

C.5 

C.6 

C.l 

C.8 

C.9 

C.10 



C.ll 
C.12 
C. 13 
C. 14 

C. 15 



C. 16 
C. 17 
C. 18 



or B.8 

DS__Try to_Start( FirstRid , FirstAttribute ) 

— /* Try - to convert locks on as many attributes as possible, for each 
request in the traffic unit, we start with the first attribute 
needed by the request and continue with other attributes needed 
by the request. 



{ 



this 



fo^ each request in the traffic unit 

rid = request id of the request in the traffic unit; 

TUAT AttrPtr = pointer to the first attribute (TUAT entry) needed 
— by this request; 

MoreAttrs = TRUE; 

ConvertFlag = TRUE; 
while ( MoreAttrs ) 

/* try to convert lock on the next attribute needed by 
request */ 

ConvertFlag = DSCC LockConversion ( rid, TUAT AttrPtr); 
if ^ConvertFlag — — 

send a message to DM to start DS on the attribute 
THAT AttrPtr->attr for the request rid; 
set TUAT AttrPtr to point to the next attribute (TUAT entry) 
needed" by this request. If it was last attribute for 
this request, set MoreAttrs to FALSE to indicate there 
are no more attributes to convert lock; 

}/* end if */ 

}/* end while */ 

}/* end for */ 



C.19 }/* end DS Try to Start */ 



B = A. 5 or 17.4 

B.l DSCC ReleaseAttr( ReleasingRid, ReleasedAttr) 

/*" This routine is used when DM sends a message to DM CC signaling 
that a request has finished DS on an attribute, and" the attribute 
can now be released. 

This routine releases the attribute and tries to start DS for 
other requests that are waiting for the attribute. */ 

B . 2 { 

B.3 remove the attribute entry from TUAT and ATUT; 

/* try to start DS for other requests */ 

B.4 for each request following the request ReleasingRid in ATUT for the 

, attribute ReleasedAttr 

B.5 { 

B.6 rid = request id of the next request in ATUT for the attribute; 

B.7 TUAT_AttrPtr = pointer to the attribute ReleasedAttr (TUAT entry) 

which is needed by the request rid; 

/* Try to convert locks on as many attributes as possible. We 
start with the attribute ReleasedAttr which is needed by the 
request rid and continue with other attributes needed by the 
request rid. 

^ We stop when we get tp an attribute that we cannot lock. */ 

B.8 DS_Try_to_Start ( rid, TUAT_AttrPtr ); 

B.9 }/* end for */ 



B.10 }/* end DSCC ReleaseAttr */ 
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A = 20.4 or 26.4 

A.l DSCC InsertUpdateReleaseAttributes ( FinishedRid ) 

7* find and remove update attribute, if any, from TUAT & ATUT */ 

A . 2 { 

A. 3 for each type-C attribute 

A. 4 { 

/* remove the attribute from TUAT & ATUT and try to start DS */ 
A. 5 D5CC_ReleaseAttr ( FinishedRid , Attribute ); 

A. 6 } /* end for */ 

A. 7 } /* end DSCC_InsertUpdateReleaseAttributes */ 



C.ll.l DSCC LockConversion ( rid, TUAT AttrPtr ) 

/*■ This routine tries to concert lock on attribute TUAT AttrPtr->attr 
for reguest rid. It returns TRUE if the lock is converted, FALSE 
otherwise. */ 

C.11.2 { 

C.11.3 ATUT ReqPtr = pointer to the request rid in ATUT for 

attribute TUAT_AttrPtr->attr 

C.11.4 if TUAT AttrPtr->mode = W then 

C.11.5 {/* tire request is asking for write access; 

the access can be granted only if 

(1) the request is the first request on the list, 

i.e., no other reguest is using the attribute 
or (2) the request is the FIRST insert-caused-by-an-update 
that is the first request on the list. */ 

C.11.6 if, first request on ATUT for attribute TUAT AttrPtr->attr 

C.11.7 {/* Case (1) ; access granted */ — 

C.11.8 return (TRUE) ; 

C.11.9 } 

C.11.10 else if FIRST insert-caused-by-update & update is first 

C.ll.ll {/* Case (2) : access granted */ 

C.11.12 return (TRUE) ; 

C.11.13 } 



C.11.14 

C.11.15 

C.11.16 



else 

{/* access cannot be granted */ 
return (FALSE) ; 



C.11.17 
C.11.18 
C. 11.19 
C. 11.20 



C.11.2I 



C.11.22 

C.11.23 

C.11.24 
C.11.25 
C.11.26 
C. 11 .27 
C.11.28 



}/* end then part of if TUAT AttrPtr->mode = W */ 
else 

{/* the request is asking for read access; the access can be 
granted only if all tne earlier requests on ATUT are also 
read accessing */ 



for all the earlier entries in ATUT for the attribute 

TUAT_AttrPtr->attr 

ATUT Ptr = pointer to the next entry in ATUT for the attribute 

TUAT AttrPtr->attr 

if ATUT_Ptr->mode = W ~ 

{/* access cannot be granted */ 
return (FALSE) ; 

}/* end if */ 

}/* end for */ 



/* all the earlier requests are also read accessing, so the 
access can be granted */ 

C.11.29 return (TRUE); 



C.11.30 }/* end else part */ 

C.11.31 }/* end DSCC LockConversion */ 
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14.1 CSCC NewTrafUnitO 

/* This routine is used when a new traffic unit and the 
descriptor- id groups needed by the requests in it are 
sent from DM to DM CC. This routine adds the traffic 
unit to TUDIGT, and" it tries to start CS. */ 

14.2 { 

/* Get the traffic unit and store it. */ 

14.3 CS_Traff ic_Unit_Init( FirstRid ); 

/* Try to convert locks on as many descriptor-id groups 
as possible. */ 

14.4 CS_Try_to_Start ( FirstRid ); 

14.5 }/* end CSCC NewTrafUnit */ 



23.1 

23.2 

23.3 

23.4 



CSCC Complete () 

— /* the descriptor-id groups needed in CS are released */ 

/* receive the request id */ 

DM_CC R$ReleaseDidG roups ( ReleasingRid) ; 

/* remove tTTe descriptor-id groups from TUDIGT, 
and try to start CS */ 

CSCC ReleaseDidGroups ( ReleasingRid); 



23.5 }/* end CSCCcomplete */ 



14.3.1 



CS Traffic Unit Init( FirstRid ) 

— /* Get "Che traffic unit and store it. */ 

/* The message contains a traffic unit and the descriptor-id groups 
needed by the requests in it. The message is 

{ [ridl, (desc-id groupli, desc-id grouplj, ...)], 

[rid2, (desc-id group2i, desc-id group2j, ...)], 

... } */ 



14.3.2 



14.3.3 

14.3.4 

14.3.5 

14.3.6 

14.3.7 



14.3.8 



/* receive the traffic unit and the descriptor-id groups */ 
DM_CC_R$Desc IdG roups ( ... ); 

/* enter the requests and their corresponding descriptor-id groups 
into TUDIGT */ 

for each request in the traffic unit 

for each descriptor-id group needed by this request 
add an entry to TUDIGT; 

/* the entry contains the following information 
descriptor-id group 
mode (R/W) 

category (TBU) */ 

}/* end for */ 



14.3.9 }/* end for */ 

14.3.10 } /* end CS Traffic Unit Init */ 
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E = 14.4 or D.4 

E.l CS_Try to_Start( FirstRid ) 

/* Try to convert locks on as many descriptor-id groups as possible 
for this and later traffic units. For each request in a traffic 
unit, we start with the first descriptor-id group needed by the 
request and continue with other descriptor-id groups needed by 
the request . 

The Cluster Search for a request can proceed when all the 
descriptor-id groups needed by the request are locked. V 

E . 2 { 

E.3 fo^ each request in this or later traffic units 

E.5 TUDIGT_ReqPtr = pointer to the next request (TUDIGT entry) in 

the traffic unit; 

/* try to convert locks on as many descriptor-id groups (needed 
by this request) as possible */ 

E.6 ConvertFlag = Request CSCC LockConversion( TUDIGT ReqPtr) ; 

E.7 if ConvertFlag 

E.8 {/* all descriptor-id groups needed by the request are locked */ 

E.9 send a message to DM to start CS for the request 

TUDIGT ReqPtr->rid; 

E. 10 }/* end if */ “ 

E.ll }/* end for */ 

E.12 }/* end CS_Try_to_Start */ 



D = 23.4 or 26.5 

D.l CSCC ReleaseDidGroups ( ReleasingRid ) 

/*■ This routine is used when DM signals DM CC that the descriptor-id 
groups corresponding to a request can be - released. 

This routine releases the descriptor-id groups and tries to start 
CS for other requests that are waiting. */ 

D.2 { 

D.3 remove TUDIGT entries corresponding to the descriptor-id groups for 

the request ReleasingRid; 



/* try to start CS for other requests */ 
D.4 CS_Try_to_Start ( next later request ); 



D.5 }/* end CSCC_ReleaseDidG roups */ 



E.6.1 Request CSCC_LockConversion( TUDIGT_ReqPtr ) 

/* This routine tries to convert locks on descriptor-id groups 
corresponding to the request TUDIGT ReqPtr. It returns TRUE if 
all the descriptor-id groups needed - by the request are locked, 
FALSE otherwise. */ 

E.6. 2 { 

/* Try to convert locks on all the descriptor-id groups needed by 
the request. We stop when we get to a descriptor-id group 
that we cannot convert lock */ 

E.6. 3 for each descriptor-id group needed by the request TUDIGT ReqPtr 

until we cannot convert lock — 

E.6. 4 { 

E.6. 5 TUDIGT ReqDidGroupPtr = pointer to the next descriptor-id group 

needed by the request TUDIGT ReqPtr; 

E.6. 6 if TUDIGT ReqDidGroupPtr->category is not BU 

E.6. 7 {/* try - to convert lock */ 

E.6. 8 ConvertFlag = CSCC_LockConversion( TUDIGT_ReqDidGroupPtr ) ; 

E.6. 9 }/* end if TUDIGT_ReqDidGroupPtr->category is not BU */ 

E.6. 10 }/* end for */ 

E.6. 11 return ( ConvertFlag ) ; 



E.6. 12 }/* end Request CSCC LockConvers ion */ 
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E.6.8.1 CSCC LockConversion( DidGroupPtr ) 

/*" This routine tries to convert locks on descriptor-id group, 
DidGroupPtr corresponding to the reguest TUDlGT ReqPtr. It 
returns TRUE if all the aescriptor-id groups needed by the 
request are locked, FALSE otherwise. */ 



E.6.8.2 { 



E.6.8.3 

E.6.8.4 

E.6.8.5 



/* Try to convert locks on all the descriptor-id groups needed by 
the request. We stop when we get to a descriptor-id group that 
we cannot convert lock */ 

fo^ each descriptor-id group needed by the request TUDIGT_ReqPtr 

HJDIGT ReqDidGroupPtr = pointer to the next descriptor-id group 

needed by the request TUDIGT ReqPtr; 



E.6.8.6 

E.6.8.7 



E.6.8.8 

E.6.8.9 

E.6.8.10 



if TUDIGT ReqDidGroupPtr->category is not BU 
{/* try - to convert lock */ 

/* check this descriptor-id group with all the descriptor-id 
groups corresponding to the EARLIER requests in TUDIGT */ 
for each descriptor-id group corresponding to EARLIER requests 

TUDIGT_Ptr = pointer to TUDIGT entry for 
this descriptor-id group 



E. 6.8. 11 
E.6.8.12 
E.6.8.13 



E.6.8.14 
E.6.8.16 
E.6.8.17 
E. 6.8. 18 



/* we have to compare the two descriptor-id groups only if 
one (or both) request is asking for write access (two 
reads can go concurrently) */ 

if ^TUDIGT_ReqDidGroupPtr->moae = W or TUDIGT_Ptr->mode = W 

if the two descriptor-id groups have conflicts, 

i.e., there can be a descriptor-id group 
. containing both groups 

{/* lock cannot be converted */ 
return (FALSE) ; 

}/* end if */ 

}/* end if */ 



E.6.8.19 



E.6.8.20 



E.6.8.21 



E.6.8.22 



E.6.8.23 



}/* end for */ 

/* We have compared TUDIGT ReqDidGroupPtr with all the earlier 
descriptor-id groups, SC the lock can be converted */ 
TUDIGT_ReqDidG roupPtr->category = BU; 

}/* end if TUDIGT_ReqDidGroupPtr->category is not BU */ 

}/* end for */ 

/* all the descriptor-id groups for the request have been locked */ 
return (TRUE) ; 



E.6.8.24}/* end CS CC LockConversion */ 
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APPENDIX C 

THE SSL SPECIFICATIONS FOR DIRECTORY MANAGEMENT 



The system specification for directory management due to concurrency con- 
trol of directory data is given in this appendix. 



/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 



Design: 

Designer 

Date 



DM 

A. Orooji 
July 7, 1983 



(1) Directory Management with Concurrency Control on Directory Data 

f ’ 

(6) Purpose 

The directory data, namely the descriptor-to-descriptor-id 
table fDDIT) and the cluster-definition table (CDT) , may be 
modified during request execution. Thus before descriptor search (DS) 
can access DDIT or cluster search(CS) can access CDT, appropriate 
locks must be obtained. Directory management was modified to allow 
for concurrency control of this directory data. 



V 

V 

V 

V 

V 

V 

V 

V 

*/ 

V 

V 



(8) Procedure Hierarchy for Directory Management(DM) 






DM 

init 



DM_R$ 

Message 
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Sender 



■H 



DM 

I 
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DESC 
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H — 
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DI DP2 
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MSy 






DM R$ 
Atfr 



DM R$ 
Rip 



I 

DM DMCC 

Attrr_ 

Locked 



+- 



I 
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Locked 
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CLUS 
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NINS 
CLUS“ 
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I 
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DM R$ 
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ADDR GR 



NINS 

ADDR-GR 



-H + 

I 

DM DBCC 

Executable 

Req 



-+ 

I 

DM S$ 
RePProc 
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DIDs 



RDT SAVE 
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(11) Program Specifications 



List of abbreviations: 

AG - address generation 

AT - attribute table 

CC - concurrency control 

COT - cluster-definition table 

CS - cluster search 

DB - database 

DDIT - descriptor-to-descriptor-id table 

DM - directory management 

DS - descriptor search 

IIG - insert information generation 



*/ 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 



iain() /* Directory-Management */ 

/* do initialization */ 

DM init ()_i 
whtle ( TRUE ) 

{ , 

/* get the next message for Directory-Management */ 

DM R$Message () ; 

/*~ get the sender name of the message */ 
sender = DM_R$Sender ( ) ; 
switch ( sender ) 

case G PCLB: /* message from controller or another backend */ 
DMJCNTL ANOTHER BE MSG () ; 
break; 



13 case THIS BACKEND: /* message from this backend */ 

14 DM THIS BE MSG (sender) ; 

15 br«Fak; 



16 default: 

17 error; 

18 break; 



19 



}/* end switch */ 



20 }/* end while */ 

21 }/* end main */ 
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11.1 



11.2 

11.3 

11.4 

11.5 

11.6 



11.7 



11.8 

11.9 

11.10 
11.11 
11.12 



11.13 

11.14 

11.15 

11.16 



11.17 

11.18 



11.19 

11.20 
11.21 



DM_CNTL ANOTHER_BE_MSG ( ) 

/* ITTis routine is used when there is a message for Directory 
/* Management from the controller or another Backend. 



/* get the type of the message */ 
MsgType = DM R$Type ( ) ; 
switch ( MsgType ) 



*/ 

V 



case ParsedTrafUnit: /* requests from Request-Preparation */ 

/* all the type-C attributes that are needed by the requests 
in the traffic unit have to be sent to DS CC together 
(at once) */ 



send all the type-C attributes needed by the traffic unit 

to DS CC; 

/* the type-C attributes needed by the traffic unit: 

. for an insert request in tne traffic unit, all the 
type-C attributes in the record (lock for write 
access) 

. for a non-insert request in the traffic unit, the 
type-C attributes in that part of the query that this 
backend will perform DS on (lock for read access) 

. for an update request, the attribute-being-modified 
if it is type-C (lock for write access) */ 

/* process the requests one by one */ 
for each request in ParsedTrafUnit 

if ( ReqType == INSERT ) 



done = INS DESC SR( ... ); 

/*“ INS "DESC SR returns true if it performs DS 
on nil tire keywords that this backend is 
supposed to perform DS on. */ 
j/* Cluster Search & Address Generation are done later */ 

else 



{/* request is non-insert */ 
done = NINS DESC SR( ... ); 

/* TUNS DESC SR returns true if it performs DS 
on aTl the" predicates that this backend is 
supposed to perform DS on. */ 

/* Cluster Search & Address Generation are done later */ 

. } 

if ( done = true ) 

/* DS is done on all keywords/predicates that this 
backend is supposed to do DS on; broadcast the 
descriptor ids to the other backends */ 

DM Broadcast DIDs( ... ); 

}/* entf for */ — 
break; 



11.22 case NewDesc: /* new descriptor from Descriptor-Id-Generator */ 

/* receive the descriptor id generated in the controller */ 

11.23 DM_R$DescId (&rid, &predicate, &new desc_id) ; 

/* store the descriptor id and indicate that DS is done for a 
keyword */ 

11.24 done = DI DP2(&rid, &predicate, &new_desc_id) ; 

/* DT DP2 returns true if DS is done on all the keywords 
that this backend is supposed to perform DS on. */ 

11.25 if ( done = true ) 

/* DS is done on all keywords that this backend is supposed 
to do DS on; broadcast the descriptor ids to the other 
backends */ 

11.26 DM Broadcast DIDs( ... ); 

11.27 breaky 
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11.28 case BeNo: /* backend number (selected for record insertion) 

from Backend-Selector */ 

/* receive the backend number V 

11.29 DM R$Be No(&rid, &be_no, &cid); 

11.30 if - this - backend is supposed to insert the record 

11.31 { 

11.32 j update directory tables if needed (new cluster) ; 

11.34 release the descriptor-id group locked by the request; 

11.35 if all the requests in the traffic unit nave finished 

Cl ust6r SGciirch 

11.36 {/* the traffic unit is sent to DB CC when all the requests 

in it have finished CS */ 

11.37 if all the earlier TUs which had conflict with this T'J 

in CS have been sent to DB CC 
/* recall: traffic units are sent iTT order to DB_CC */ 

11.38 send the traffic unit to DB Concurrency-Control; 

11.39 } 

11.40 break; 

11.41 case ... 

11.42 }/* end switch */ 

11.43 }/* end DM CNTL ANOTHER BE MSG */ 



14.1 DM_THIS BE_MSG (sender) 

/* TfTis routine is used when there is a message for Directory 
Management from a task in the same backend. */ 

14.2 { 

14.3 switch ( sender ) 

14.4 { 

14.5 case DM ConcurrencyControl : 

14.6 DM DMCC MSG(); 

14.7 br^ak; - 

14.8 case DB ConcurrencyControl: 

14.9 DM DBCC MSG(); 

14.10 break; - 

14.11 case Record Processing: 

14.12 break; 

14.13 default: 

14.14 error; 

14.15 break; 

14.16 }/* end switch */ 

14.17 }/* end DM THIS BE MSG */ 
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14.6.1 



14.6.2 

14.6.3 

14.6.4 

14.6.5 

14.6.6 

14.6.7 

14.6.8 



14.6.9 



14.6.10 

14.6.11 



DM DMCCJMSG ( ) 

7* This routine is used when there is a message for Directory */ 
j /* Management from DM_Concurrency Control (in the same backend). */ 

/* get the type of the message */ 

MsgType = DM R$Type ( ) ; 

^ switch (“'MsgType ) 

case AttrLocked: /* attr needed by a request is locked */ 

/* get the attribute name */ 

DM R$Attr(&rid, attr); 

/* do the descriptor search if needed */ 
done = DM DMCC Attr Locked (&rid, attr) 

/*“ DM DMCC ATitr Locked returns true if DS is done. on 
alT the - keyWdrds/predicates that this backend is 
supposed to perform DS on. */ 
if ( done = true ) 

/* DS is done on all keywords/predicates that this backend 
is supposed to do DS on; broadcast the descriptor ids 
to the other backends */ 

DM_Broadcast_DIDs( ... ); 
break; 



14.6.12 

14.6.13 

14.6.14 

14.6.15 



case DidGroupsLocked : /* descriptor-id groups needed by a request 

are locked */ 

/* receive the request id */ 

DM R$Rid(&rid); 

/* 'do the Cluster Search */ 

DM DMCC DidGroups Locked (&rid) ; 
break; — 



14.6.16 



CdSG • • • 



14.6.17 }/* end switch */ 

14.6.18 }/* end DM DMCC MSG V 



14.9.1 



14.9.2 



14.9.3 

14.9.4 

14.9.5 

14.9.6 



14.9.7 

14.9.8 

14.9.9 



DM DBCC MSG () 

7* ThTs routine is used when there is a message for Directory */ 
/* Management from DB_Concurrency Control (in the same backend) . */ 

/* get the type of the message */ 

MsgType = DM R$Type ( ) ; 
switch ( MsgType ) 

case ExecutableReq: /* DB CC has given permission to execute 

request { i.e., the - request can continue with Address 
Generation and the rest of request execution */ 

/* receive the request id */ 

DM R$Rid (&rid) ; 

/* <?o the Address Generation */ 

DM_DBCC_ExecutableReq(&rid) ; 
break; 



14.9.10 



case . . . 



14.9.11 }/* end switch */ 

14.9.12 }/* end DM DBCC MSG */ 
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11.12.1 INS DESC SR ( ... ) 

7* This routine is used for processing insert requests. It finds 
/* the descriptors that satisfy the keywords in an insert request. 



*/ 

*/ 

V 



11 . 12 , 

11 . 12 , 

11 . 12 . 

11 . 12 . 

11 . 12 . 

11 . 12 . 

11 . 12 . 

11 . 12 . 



.4 

.5 

.6 

,7 

,8 



V 

*7 



/* if there are X keywords (in the record being inserted) and N 
/* backends, each backend performs descriptor search on X/N 
/* keywords . , , 

11.12.3 for each keyword that this backend is supposed to do descriptor 

search 

if attr in keyword is not type C 
do the descriptor search; 

/* type-C attributes should be locked for write before DS */ 
}/* end for V 

if DS is done on all keywords that this backend is supposed to do 

DS on 

9 return (true) ; 

10 else 

11 return (false) ; 



11.12.12 }/* end INS DESC SR */ 



11.16.1 NINS DESC SR ( ... ) 

/*■ This - routine is used for processing non-insert requests. 
/* finds the descriptors that satisfy the predicates in a 
/* non-insert request. 



It 



11.16.2 { 



11.16.3 



*/ 

V 

V 



11.16. 

11.16. 

11.16. 

11.16. 

11.16. 

11.16. 

11.16. 

11.16. 



4 

,5 

6 

7 

8 

9 

10 
11 



/* if there are X predicates (in the query) and N backends, each */ 
/* backend performs descriptor search on x/N predicates . */ 

for each predicate that this backend is supposed to do descriptor 
^ search 

if attr in predicate is not type C 
do the descriptor search; 

/* type-C attributes should be locked for read before DS */ 
}/* end for V 

if DS is done on all predicates that this backend is supposed to do 
DS on 

return (true) ; 
else 

return (false) ; 



11.16.12 }/* end NINS DESC SR */ 



11.24.1 DI DP2 (rid, predicate, new desc id) 

/* This routine is used - for Tnsert requests. It is called when */ 
/* Insert-Information-Generation sends a new descriptor id to the */ 
/* backends. Ibis routine adds the descriptor id to DDIT. It */ 
/* also indicates that the descriptor id is ready for the insert */ 
/* request which is waiting for tne descriptor ia. */ 

11.24.2 { 

11.24.3 update the DDIT (and AT if needed); 

11.24.4 for the insert request which is waiting for this descriptor id f 

indicate the descriptor search for that keyword is finished; 

11.24.5 if DS is done on all keywords that this backend is supposed to do 

DS on 

11.24.6 return (true) ; 

11.24.7 else 

11.24.8 return (false); 



/* the type-C attributes will be released before Cluster Search */ 

11.24.9 }/* end DI DP2 */ 
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14.6.8.1 



14.6.8.2 

14.6.8.3 

14.6.8.4 

14.6.8.5 

14.6.8.6 



*/ 

V 

V 



14.6.8.7 

14.6.8.8 
14.6.8.9 



DM DMCC_Attr Locked (rid, attr) 

/*■ This routine is used when DM Concurrency Control signals 
/* Directory Management that an - attribute needed by a request 
/* is locked. 

{ 

ReqType = type of the request; 
switch ( ReqType ) 

case INSERT: 

/* we recall that this backend locks all type-C attributes in 
the record even those that other backends will do DS on, so 
we need to check to see if this backend is supposed to do 
DS; we can have DM Concurrency Control not send a message 
back for the attributes that will be processed in the other 
backends [Directory Management, of course, must tell 
DM Concurrency Control (when sending attributes) about this] 
if^tlTis backend is supposed to do descriptor search 

do descriptor search for the keyword having the attribute; 
/* if the type-C attribute has a new value, we need a new 
descriptor. In descriptor search, if no descriptor is 
found, a message is sent to the Insert-Information- 
Generation to generate a new descriptor id. */ 



V 



14.6.8.10 

14.6.8.11 

14.6.8.12 

14.6.8.13 

14.6.8.14 



14.6.8.15 



if descriptor search is done on all keywords 
return (true) ; 
else 

return (false) ; 

}/* end if */ 

/* the type-C attributes will be released before 
cluster search */ 

break; 



14.6.8.16 

14.6.8.17 



14.6.8.18 



14.6.8.19 

14.6.8.20 

14.6.8.21 

14.6.8.22 

14.6.8.23 

14.6.8.24 



case RETRIEVE: 
case DELETE: 

/* we recall that, for non-insert requests, only the attributes 
that this backend is supposed to do descriptor search on 
are sent to DM Concurrency Control */ 
do the descriptor search for all predicates having 
the attribute; 
release the attribute; 

if descriptor search is done on all predicates 
return (true) ; 
else 

return (false) ; 
break; 



14.6.8.25 

14.6.8.26 

14.6.8.27 

14.6.8.28 

14.6.8.29 

14.6.8.30 

14.6.8.31 

14.6.8.32 

14.6.8.33 

14.6.8.34 

14.6.8.35 



case UPDATE: 

do the descriptor search for all predicates having 
the attribute; 

if attr is not the attribute-being-modified 

release the attribute since DDIT will not be modified; 
else 

don't release the attribute yet since DDIT may be modified 
as a result of inserts caused by the update; 
if descriptor search is done on all predicates 
return (true) ; 
else 

return (false) ; 
break; 



14.6.8.36 default: 

14.6.8.37 error; 

14.6.8.38 break; 

14.6.8.39 }/* end switch */ 

14.6.8.40 }/* end DM DMCC Attr Locked */ 
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14.6.14.1 DM DMCC DidGroups Locked (rid) 

/*" Thi'S' routine Ts used when DM Concurrency Control signals 
/* Directory Management that the' descriptor-id groups needed by 
/J ' request are locked. The request can then proceed with 



14.6.14.2 

14.6.14.3 

14.6.14.4 

14.6.14.5 



{ 



/* . 

/* Cluster Search. 



ReqType = type of the request; 
switch ( ReqType ) 



% 

V 

V 



/ 



14.6.14.6 



14.6.14.7 



14.6.14.8 



14.6.14.9 



case INSERT: 

/* do Cluster Search (i.e. f find the id of the cluster to */ 
/* which the record being inserted belongs) */ 

cid = INS_CLUS_GR( ... ) ; 

/* the descriptor-id group locked by the request will be 
released when cluster search is done and Insert- 
Information-Generation has determined whether or not 
there is a new cluster and CDT has been modified */ 

/* send the cluster id to Backend-Selector */ 

DM_S$BS( ... ) ; 

/* traffic unit will be sent to DB Concurrency-Control 
after Insert-Information-Generafion responds */ 

break; 



14.6.14.10 

14.6.14.11 



14.6.14.12 

14.6.14.13 

14.6.14.14 

14.6.14.15 

14.6.14.16 



14.6.14.17 

14.6.14.18 

14.6.14.19 



case RETRIEVE: 
case DELETE: 

/* do Cluster Search (i.e. { find the ids of the clusters */ 

/* that satisfy the query in the request) */ 

NINS_CLUS GR ( ... ) ; 

release the descriptor-id groups locked by the request; 
if all the requests in the traffic unit have finished 
Ol us tor Search 

{/* the traffic unit is sent to DB CC when all the requests 
in it have finished CS "V 

if all the earlier TUs which had conflict with this TU 
in CS have been sent to DB CC 
/* recall: traffic units are sent in order to DB CC */ 
send the traffic unit to DB_Concurrency-ControT; 

break; 



14.6.14.20 



14.6.14.21 

14.6.14.22 

14.6.14.23 

14.6.14.24 



14.6.14.25 

14.6.14.26 

14.6.14.27 



case UPDATE: 

/* do Cluster Search (i.e.< find the ids of the clusters 
/* that satisfy the query in the request) */ 

NINS CLUS_GR( ... ); 

7* descriptor-id groups locked by the request will be 
released after the update is completely dpne */ 
if all the requests in the traffic unit have finished Cluster 
Search 

{/* the traffic unit is sent to DB CC when all the requests 
in it have finished CS ~*/ 

if all the earlier TUs which had conflict with this TU in 
CS have been sent to DB CC 

/* recall: traffic units are sent in order to DB CC */ 
send the traffic unit to DB_Concurrency-ControT; 

break; 



14.6.14.28 

14.6.14.29 

14.6.14.30 

14.6.14.31 

14.6.14.32 



default: 

error; 

break; 

}/* end switch */ 

}/* end DM DMCC DidGroups Locked */ 
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14 . 



14. 

14. 

14. 

14. 

14. 

14. 

14. 

14. 

14. 

14. 



14. 

14. 



A = 
A. 1 



A. 2 
A. 3 

A. 4 

A. 5 

A. 6 



9.8.1 DM DBCCJSxecutableReq(rid) 

7 * This routine is used when DB Concurrency Control signals */ 
/* Directory Management that a Tequest can proceed with */ 

/* Address Generation and the rest of request execution. */ 

9.8.2 { 

9.8.3 ReqType = type of the request; 

/* perform the Address Generation for the request */ 

9.8.4 if t ReqType = INSERT ) 

9.8.5 {/* insert request */ 

9.8.6 INS ADDR GR ( ... ); 

9.8.7 } - “ 

9.8.8 else 

9.8.9 {/* non-insert request */ 

9.8.10 NINS ADDR GR ( ... ); 

9.8.11 } 

/* send the request to Record Processing */ 

9.8.12 DM_S$RecProc ( ... ); 

9.8.13 }/* end DM DBCC_ExecutableReq */ 



11.19 or 11.26 or 14.6.10 
DM_Broadcast_DIDs (rid) 

/* This routine broadcasts the descriptor ids found by this backend */ 
/* to the other backends. It also saves these descriptor ids */ 

/* [which are in request-descriptor table (RDT)] to be used in */ 

/* cluster search later. */ 

{ 

find the RDT for the request; 

/* save the RDT */ 

RDT_SAVE (rid, RDT) ; 

broadcast the descriptor ids to the other backends; 

}/* end DM Broadcast DIDs */ 
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A.4.1 



A. 4. 2 
A.4.3 

A.4.4 

A.4.5 

A.4.6 

A.4.7 

A.4.8 



A.4.9 
A.4.10 
A. 4 .11 

A. 4 .12 

A.4.13 

A.4.14 



A. 4. 15 



RET SAVE (rid, ROT) 

7* This routine saves the descriptor ids [which are in request- */ 
/* descriptor table (ROT)] found by a backend (during Descriptor */ 
/* Search) . */ 

/* This routine is called when this backend has finished DS or */ 
/* another backend has sent the descriptor ids that it has found */ 
/* in DS. V 

/* When the descriptor ids found by all the backends have been */ 
/* received and saved, we can proceed to Cluster Search. */ 

save the ROT; 

/* check to see if all the backends have sent the descriptor ids */ 
if all ROTs for the request have been saved 
{/* the request has finished DS */ 
if the request is insert 

release all the type-C attributes locked by the request; 

/* we recall that for non-insect requests, each type-C 
attribute is released immediately after DS is aone 
on that attribute */ 

} 

if all requests in traffic unit have finished Descriptor Search 
{/* the traffic unit is sent to CS CC when all the requests 
in it have finished DS */ — 

if all the earlier TUs which had conflict with this 111 in DS 

have been sent to CS_CC 

/* recall that traffic units are sent in order to CS CC */ 



send the traffic unit to CS_Concurrency-Control; 

/* the descriptor-id groups corresponding to the requests 
in the traffic unit are sent to DM Concurrency-Control 
(for read/write access depending oTT the request types); 
DM_Concurrency-Control will respond when all the 
descriptor-id groups for a request have been locked. At 
that time. Cluster Search for that request can be 
performed */ 



A. 4. 16 }/* end if all the requests in the TU have finished DS */ 

A. 4. 17 }/* end if all the ROTs for the request have been saved */ 

A. 4. 18 }/* end ROT SAVE */ 
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APPENDIX D : REMAINING ALGORITHMS FOR THE SECONDARY-MEMORY-BASED 



DIRECTORY MANAGEMENT 

This appendix contains descriptions of the remaining algorithms for the 
secondary-memory-based directory management. The first section examines what 
happens when the directory data is modified. The second section explains how 
to determine if an updated record has changed cluster. 



D. 1_ Updating the Directory Data 

In this section, we explain the algorithms used to update the directory 
data. We examine two types of updates: one is due to the fact that a new 
type-C sub-descriptor is defined, and the other when a new cluster is defined. 
When a new type-C sub-descriptor is defined, the DDIT and the DCBMT must be 
modified. This is covered in Section D.1.1. When a new cluster is defined, 
the DCBMT is modified. We review this procedure in Section D.1.2. 



D.1.1 Updating the DDIT and the DCBMT when a New Descriptor is Defined 

As was described in Section 4.2, new descriptors are not inserted until 
descriptor search is finished for all the descriptors in the request. At that 
time, both the descriptor-to-descriptor-id table (DDIT) and the descriptor- id- 
cluster-bit-map table (DCBMT) must be updated. This processing is described in 
the next two sections. 



(A) Updating the DDIT 

The states and transitions for inserting a new descriptor into the DDIT 
are shown in Figure D.l. The descriptor information added to the DDIT con- 
sists of a descriptor id and a range of values which specify the new descrip- 
tor. The first step in inserting a new descriptor is to read the appropriate 
sequence node (5). (We recall that this sequence node was identified in the 
descriptor search phase. When the descriptor search for this descriptor was 
unsuccessful, the pointer to the sequence node where the new descriptor infor- 
mation should be inserted was then determined.) If there is room in this node, 
then the descriptor is inserted(17) and processing is finished (18) . If there 
is no room in this node, the node must be split and both halves written to 
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Note : All reads and writes in the state diagram are for the DDIT 
unless otherwise specified. 



Figure D.l. Inserting a New Descriptor into the DDIT 
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disk (6, 7). Then a new entry must be added to the parent index node, which 
must first be read(l). If the parent index node is also full, then it will 
also have to be split and an entry added to its parent. In general, it may be 
necessary to add a new entry and split several parent index nodes (11, 12, 13) . 
Once an index node is found that is not full, the last insertion is made (15) 
and processing terminates for that descriptor (16) . In the worst case, the 
root node will be full. This occurs when the root node is either an index or 
a sequence node. In either case, the root node must be split and a new root 
node created (14 for index, 8 for sequence). In addition, since the attribute 
table contained a pointer to the old root node, the attribute table must also 
be modified (9,3) . Then the processing is finished(4). This procedure is used 
when the DDIT exists for the given attribute. 

There is a special case to consider when the DDIT does not exist for the 
given attribute. In this case, we are inserting the first descriptor for the 
given attribute. When the first descriptor for an attribute is inserted, it 
is only necessary to create the root node(l), which is a sequence node, and 
link the attribute table to it as before(2,3) . This completes the processing 
for this case (4). After the descriptor has been added to DDIT, we must add 
the descriptor to the DCBMT. 

(B) Updating the DCBMT 

The states and transitions for modifying the DCBMT when a new descriptor 
is defined are given in Figure D.2. The bit map being added for a descriptor 
specifies which cluster (s) contain the descriptor. To find the correct place 
to insert the new descriptor into the DCBMT, we work with the descriptor id. 
Recall that descriptor ids are in the form (attribute id, descriptor-with in- 
attribute) . This pair is converted into a single descriptor number. These 
numbers are then subdivided into groups. For each group there is a bit-map 
set. When processing a new descriptor id, there may already be a bit-map set 
for the group which contains the new descriptor id.. If the bit-map set 
exists, it must be read (8) and updated (9). If the bit-map set does not exist, 
a bit-map set must be created (3). In either case, the bit map itself must be 
created. This bit map is subdivided into several blocks, each of which must 
be initialized and stored on the secondary memory(5) . The last bit-map block 
is special (6) because after it is written, the insertion of this descriptor is 
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Figure D.2. Inserting a New Descriptor into the DCBMT 
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finished (7) . 



Two special cases must also be considered. First there may not yet be 
any database and therefore there will be no clusters. In this case, the bit- 
map set may or may not already exist. If the bit-map set does not exist, one 
is created (1). If the bit-map set does exist, it must be read (8) and 
updated (10). In either event, since there are no bit-map blocks for the new 
bit-map set, processing is done (2). The second special case occurs when there 
are only a few clusters so that there is only one bit-map block for each 
descriptor. In this case, after the bit-map set is read (8) and updated (9), 
there is only one bit-map block to be written(ll) , which completes the pro- 
cessing (7) . 



D.1.2 Updating the DCBMT when a New Cluster is Defined 

When a new cluster is created by an insert request, it is necessary to 
add the new cluster id and corresponding descriptor-id set to the DCBMT. This 
addition occurs after the backend number at which the insert request will be 
executed is received from the controller. At this time a new column, 
corresponding to the new cluster id, must be added to the DCBMT. Thus a 1 bit 
has to be added to the bit map for each descriptor id in the descriptor-id set 
corresponding to the cluster. Ihe bit maps for the other descriptor ids do 
not have to be modified, since all yet to be used entries were set to 0 when 
those bit maps were created. 

Figure D.3 shows the states and transitions needed to insert the entry 
for one descriptor id into the DCBMT. These steps must be repeated for each 
descriptor id in the descriptor-id set corresponding to the new cluster. The 
first step is to read the bit-map set corresponding to the descriptor id(l). 
Then the first block of the bit map is read (5), followed by the rest of the 
bit-map blocks (6). Now there are two cases depending on whether or not there 
is space in the last bit-map block. If there is space in the last bit-map 
block, then the appropriate bit is changed to 1 and this block is written(9). 
On the other hand, if there is not space in the last bit-map block, then a new 
block must be created and linked to the previous last block. Thus a new 
secondary storage address is obtained and this address is put in the previous 
last block, which is then written(7). Then the new block is written(8). In 
either case, processing is finished for this descriptor id after the last 
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bit-map block has been written(4). There is also one special case to con- 
sider. The cluster being added may be the first cluster in the database. In 
this case there will be no bit-map blocks. Thus the first bit-map block is 
created and linked to the bit-map set. The updated bit-map set is written (2) 
and so is the bit-map block (3), finishing the processing (4) . 

The above procedure is repeated for each descriptor id in the 
descriptor-id set corresponding to the cluster being defined. When this has 
been done, the DCBMT has been updated to contain the information for the new 
cluster. 



D.2^ Determining if an Updated Record Has Changed Cluster 

When record processing updates a record, it must find out if the record 
has changed cluster. If it has, then an insert request must be generated so 
that the record can be inserted into the correct cluster. Otherwise, the 
record is updated in place. Record processing must send a message to direc- 
tory management asking if the record has changed cluster. Directory manage- 
ment then must reply yes or no. If the attribute being modified is not a 
directory attribute, then it is clear that the record has not changed cluster. 
In addition, if the attribute being modified is a type-C attribute, then the 
record has changed cluster only if the old and new values are different. If 
they are the same, then the record has not changed cluster. In either case, 
it is not necessary to consult the directory data to determine if the record 
has changed cluster. 

In the event the attribute being modified is a directory attribute which 
is not of type C, processing is more complex. In this case we must determine 
if the old and new values are derivable from the same descriptor id. Thus we 
must determine the descriptor ids for the old and new values. This determina- 
tion requires searching the DDIT twice, once for each value. 

The states and transitions to determine if an updated record has changed 
cluster is shown in Figure D.4 The first step in determining the descriptor id 
corresponding to the old value is to read the root node(l). Then other nodes 
of DDIT are read (2) until the appropriate sequence node is found. After the 
descriptor id corresponding to the old value has been determined, the search 
for the descriptor id corresponding to the new value begins by reading the 
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root node again (3). Then other DDIT nodes are read (4) until the sequence node 
containing the descriptor id corresponding to the new value is found. After 
determining if the old and new descriptor ids are the same, a message is sent 
to record processing saying whether or not the record has changed cluster, 
completing the processing of this changed-cluster message (5) . 

One update request may cause several database records to be updated. 
Record processing must determine if each has changed cluster. Thus one update 
request may cause several messages asking if a record has changed cluster to 
be sent to the directory management. Only one of these is processed at a 
time. Others must wait. Thus after the directory management has determined 
whether one record has changed cluster, it should answer the same question for 
the next record, if any, that is waiting. 
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